Bei der Gruppe, für die Sie eine Mitteilung verfassen, handelt es sich um eine Usenet-Gruppe. Wenn Sie in dieser Gruppe Nachrichten posten, ist Ihre E-Mail-Adresse für jeden im Internet sichtbar
Funny, I posted something similar on the http://sla.ckers.org/forum/read.php?2,13209,13314#msg-13314 because when I tried to follow the link from this board the hash-based vector had been detected. But apparently it was just a Google Group URL encoding interference (see the slac.kers link above for working URLs) ;)
I built in a basic converter to detect concatenations and deal with them - I repeat: It's very basic. So what do you guys think we could do to solve following problems?:
- various types of concatenation - maybe combined with charcode - detection of charcode built together by advanced arithmetic operations including parenthesis and floats
Thx and Greetings, .mario
On Jul 4, 11:19 pm, Giorgio Maone <giorgio.ma...@gmail.com> wrote:
> Funny, I posted something similar on thehttp://sla.ckers.org/forum/read.php?2,13209,13314#msg-13314 > because when I tried to follow the link from this board the hash-based > vector had been detected. > But apparently it was just a Google Group URL encoding interference > (see the slac.kers link above for working URLs) ;)
On Jul 5, 12:00 am, ".mario" <Mario.Heider...@googlemail.com> wrote:
> - various types of concatenation - maybe combined with charcode > - detection of charcode built together by advanced arithmetic > operations including parenthesis and floats
> On Jul 5, 12:00 am, ".mario" <Mario.Heider...@googlemail.com> wrote:
> > - various types of concatenation - maybe combined with charcode > > - detection of charcode built together by advanced arithmetic > > operations including parenthesis and floats
Well - I thought about implementing a JavaScript runtime and discussed that with Kishor via chat. But the problem is - if we'd implement that we'd also have to implement one or several SQL sanddboxes to check on the SQL injections, file system stubs to check on the directory traversal and so on. Furthermore we'd loose the basic principle of the application - its light weight.
Also the PHPIDS is all about enumerating badness - and weighting and assessing them. The only chance we have to deliver a value like the impact is to keep a library of possibilities. Of course, the regular expressions need optimizations here and there but I am still the opinion that this is the right way. Imagine the situation we are using a JS interpreter and one vector slips through - we'd need weeks to file the bug to the developers, wait for patches, test ourselves etc. The rules are fixed and committed within hours. Comparable thing with the halfwidth/fullwidth issue and the hardware IDS systems - we were able to detect that input one hour after being posted on sla.ckers/ in the sec news.
Of course we have a kind of arms race at the moment to keep up with your awesome findings but isn't that what it's all about? I've been developing with JS for years and never learned that much about the basic fundamental principles and - let's call them features - of this language. I luv it! I've never seen XSS vectors that advanced like in the last three days when Sir Darck Cat and you appeared in the group and I guess I am not alone with that attitude.
So my proposal is to keep the mentioned principles of the PHPIDS, work out the current issues (has anyone studied math in here?) and keep enhancing the software. Btw - it goes without saying that you guys will be credited in the release notes for the coming release as well as on the contact/credits page of php-ids.org ;)
Also I am planning to write a rather detailed and in depth blog post about advanced filter evasion for gnucitizen.org and I would like to use some of the vectors as examples (the knowledge we gathered here has to be persisted) - you're ok with that?
> On 4 jul, 18:08, Giorgio Maone <giorgio.ma...@gmail.com> wrote: > > On Jul 5, 12:00 am, ".mario" <Mario.Heider...@googlemail.com> wrote:
> > > - various types of concatenation - maybe combined with charcode > > > - detection of charcode built together by advanced arithmetic > > > operations including parenthesis and floats
> > At least, that's how I implemented NoScript "light" filters... a lot > > less headaches, and most important, no "enumerating badness" > > madness :)http://www.google.it/search?q=enumerating+badness
This could be an example.. $real_code=evalJS("try{ return new Function(unescape('".rawurlencode($input_code)."')).toSource(); } catch(e) { return e.description; }");
That will return the "real code" of the function, and wont execute it.. the real code I mean, that it will delete useless parenthesis, useless white spaces, etc.. and that should make easier for checking with regular expressions.
Any way, I think that with some work the regular expressions will work as well, and wont need the client to have access to the passthru function.
About the:
> Also I am planning to write a rather detailed and in depth blog post about > advanced filter evasion for gnucitizen.org and I would like to use some of > the vectors as examples (the knowledge we gathered here has to be persisted) > - you're ok with that?
Yes of course.
Greetings!!
On 5 jul, 02:55, "Mario Heiderich" <mario.heider...@googlemail.com> wrote:
> Well - I thought about implementing a JavaScript runtime and discussed that > with Kishor via chat. But the problem is - if we'd implement that we'd also > have to implement one or several SQL sanddboxes to check on the SQL > injections, file system stubs to check on the directory traversal and so on. > Furthermore we'd loose the basic principle of the application - its light > weight.
> Also the PHPIDS is all about enumerating badness - and weighting and > assessing them. The only chance we have to deliver a value like the impact > is to keep a library of possibilities. Of course, the regular expressions > need optimizations here and there but I am still the opinion that this is > the right way. Imagine the situation we are using a JS interpreter and one > vector slips through - we'd need weeks to file the bug to the developers, > wait for patches, test ourselves etc. The rules are fixed and committed > within hours. Comparable thing with the halfwidth/fullwidth issue and the > hardware IDS systems - we were able to detect that input one hour after > being posted on sla.ckers/ in the sec news.
> Of course we have a kind of arms race at the moment to keep up with your > awesome findings but isn't that what it's all about? I've been developing > with JS for years and never learned that much about the basic fundamental > principles and - let's call them features - of this language. I luv it! I've > never seen XSS vectors that advanced like in the last three days when Sir > Darck Cat and you appeared in the group and I guess I am not alone with that > attitude.
> So my proposal is to keep the mentioned principles of the PHPIDS, work out > the current issues (has anyone studied math in here?) and keep enhancing the > software. Btw - it goes without saying that you guys will be credited in the > release notes for the coming release as well as on the contact/credits page > of php-ids.org ;)
> Also I am planning to write a rather detailed and in depth blog post about > advanced filter evasion for gnucitizen.org and I would like to use some of > the vectors as examples (the knowledge we gathered here has to be persisted) > - you're ok with that?
> > On 4 jul, 18:08, Giorgio Maone <giorgio.ma...@gmail.com> wrote: > > > On Jul 5, 12:00 am, ".mario" <Mario.Heider...@googlemail.com> wrote:
> > > > - various types of concatenation - maybe combined with charcode > > > > - detection of charcode built together by advanced arithmetic > > > > operations including parenthesis and floats
> > > At least, that's how I implemented NoScript "light" filters... a lot > > > less headaches, and most important, no "enumerating badness" > > > madness :)http://www.google.it/search?q=enumerating+badness
According to me, there are several things to consider here.
1) Implementing a JavaScript runtime: This idea may initially sound really cool, but I am of the opinion that it will not be very useful. There exist 10 different browsers and we need one for each of them. Also having a SQL sandbox is an overkill.
Let me first quote my interpretation of the use of sandbox here, if (thats correct) { please continue reading.
}
else { please explain what it is -again :) goto next_reply;
}
Here it goes: "a) After IDS receives an input, we pass the input to the sandbox and determine if there is a valid JS code in the input. We also determine if there is any JS at all. b) Other alternative: Before the response is sent to the client, we pass it trhough the sandbox, determine the JS that potentially got added and either block the response or filter the new JS
"
If any of the above statements is true, the important thing to consider is how an input is mapped to the output? In case a), input many times is not going to be in valid JS syntax. E.g. if I am injecting inside a script tag, the input looks like "; alert(1); which isn't valid JS, JS interpreter will say there isn't a valid JS code.
In case b), we have no way to know which code was added by an application and which by attacker. We can't assume that attacker will always inject alerts and things to exploit/discover XSS. So looking inside the sandbox for evidence of alert etc. is not appropriate. (I have concerns with .NETIDS)
There can sometimes be conversions applied by the app on input which makes it vulnerable. In a hypothetical example if app converts 1 to 'on' and 0 to off, then I can inject 1mouseover and injection succeeds. We can't do much in these situations.
On math question, I am not good at math but there exist so many combinations that we will not be able to handle all of them. n inputs, m encodings, l browsers, alteast 2 languages (J/V S), k places where input is echoed, p weird things inside browsers.
So conclusion is that even sandbox may not help much. Then why invest so much into it. And why make PHPIDS heavier. Really speaking I would like it to remain RegEx based. Thats the way I love it.
We should cover as many cases with regexes as possible I agree, but if we can't promise a complete solution using SandBox then why bother using it.
-Kishor
On Jul 5, 11:06 am, SirDarckCat <sirdarck...@gmail.com> wrote:
> This could be an example.. > $real_code=evalJS("try{ return new > Function(unescape('".rawurlencode($input_code)."')).toSource(); } > catch(e) { return e.description; }");
> That will return the "real code" of the function, and wont execute > it.. the real code I mean, that it will delete useless parenthesis, > useless white spaces, etc.. and that should make easier for checking > with regular expressions.
> Any way, I think that with some work the regular expressions will work > as well, and wont need the client to have access to the passthru > function.
> About the:
> > Also I am planning to write a rather detailed and in depth blog post about > > advanced filter evasion for gnucitizen.org and I would like to use some of > > the vectors as examples (the knowledge we gathered here has to be persisted) > > - you're ok with that?
> Yes of course.
> Greetings!!
> On 5 jul, 02:55, "Mario Heiderich" <mario.heider...@googlemail.com> > wrote:
> > Hi Giorgio!
> > Well - I thought about implementing a JavaScript runtime and discussed that > > with Kishor via chat. But the problem is - if we'd implement that we'd also > > have to implement one or several SQL sanddboxes to check on the SQL > > injections, file system stubs to check on the directory traversal and so on. > > Furthermore we'd loose the basic principle of the application - its light > > weight.
> > Also the PHPIDS is all about enumerating badness - and weighting and > > assessing them. The only chance we have to deliver a value like the impact > > is to keep a library of possibilities. Of course, the regular expressions > > need optimizations here and there but I am still the opinion that this is > > the right way. Imagine the situation we are using a JS interpreter and one > > vector slips through - we'd need weeks to file the bug to the developers, > > wait for patches, test ourselves etc. The rules are fixed and committed > > within hours. Comparable thing with the halfwidth/fullwidth issue and the > > hardware IDS systems - we were able to detect that input one hour after > > being posted on sla.ckers/ in the sec news.
> > Of course we have a kind of arms race at the moment to keep up with your > > awesome findings but isn't that what it's all about? I've been developing > > with JS for years and never learned that much about the basic fundamental > > principles and - let's call them features - of this language. I luv it! I've > > never seen XSS vectors that advanced like in the last three days when Sir > > Darck Cat and you appeared in the group and I guess I am not alone with that > > attitude.
> > So my proposal is to keep the mentioned principles of the PHPIDS, work out > > the current issues (has anyone studied math in here?) and keep enhancing the > > software. Btw - it goes without saying that you guys will be credited in the > > release notes for the coming release as well as on the contact/credits page > > of php-ids.org ;)
> > Also I am planning to write a rather detailed and in depth blog post about > > advanced filter evasion for gnucitizen.org and I would like to use some of > > the vectors as examples (the knowledge we gathered here has to be persisted) > > - you're ok with that?
> > > On 4 jul, 18:08, Giorgio Maone <giorgio.ma...@gmail.com> wrote: > > > > On Jul 5, 12:00 am, ".mario" <Mario.Heider...@googlemail.com> wrote:
> > > > > - various types of concatenation - maybe combined with charcode > > > > > - detection of charcode built together by advanced arithmetic > > > > > operations including parenthesis and floats
> > > > At least, that's how I implemented NoScript "light" filters... a lot > > > > less headaches, and most important, no "enumerating badness" > > > > madness :)http://www.google.it/search?q=enumerating+badness
I think a complete runtime is absolute overkill and I think we are already verging on too much interpretation of the JavaScript. I think we should focus on detecting ways of executing a payload rather than getting too hung up on the payload itself. In other words, what we can detect with regex, we detect, what we cannot we just accept that it is never going to cath everything without a runtime.
For example: How can any of the JavaScript be used if it is impossible to inject into a script tag or attribute? If we are catching attempts to inject the payload in the first place and giving it a high impact rating then it will be impossible to actually execute the payload anyway and any other information we can glean about the payload is merely an impact raiser.
I too have concerns about the implementation of output scanning in .NETIDS - it is *too* prone to false positives at the moment which is why, in next weeks release, I am going to remove it by default but also leave the capability in there as I think it is a useful tool - if only for debugging purposes to see where there is potential for fragmented injection. The output scanning works on a much smaller ruleset than the main scanner but if you have any other suggestions for how best to deal with this then I would like to hear it.
> According to me, there are several things to consider here.
> 1) Implementing a JavaScript runtime: This idea may initially sound > really cool, but I am of the opinion that it will not be very useful. > There exist 10 different browsers and we need one for each of them. > Also having a SQL sandbox is an overkill.
> Let me first quote my interpretation of the use of sandbox here, > if (thats correct) > { > please continue reading. > } > else > { > please explain what it is -again :) > goto next_reply; > }
> Here it goes: "a) After IDS receives an input, we pass the input to > the sandbox and determine if there is a valid JS code in the input. We > also determine if there is any JS at all. > b) Other alternative: Before the response is sent to the client, we > pass it trhough the sandbox, determine the JS that potentially got > added and either block the response or filter the new JS
> "
> If any of the above statements is true, the important thing to > consider is how an input is mapped to the output? > In case a), input many times is not going to be in valid JS syntax. > E.g. if I am injecting inside a script tag, > the input looks like "; alert(1); which isn't valid JS, JS interpreter > will say there isn't a valid JS code.
> In case b), we have no way to know which code was added by an > application and which by attacker. We can't assume that attacker will > always inject alerts and things to exploit/discover XSS. So looking > inside the sandbox for evidence of alert etc. is not appropriate. (I > have concerns with .NETIDS)
> There can sometimes be conversions applied by the app on input which > makes it vulnerable. In a hypothetical example if app converts 1 to > 'on' and 0 to off, then I can inject 1mouseover and injection > succeeds. We can't do much in these situations.
> On math question, I am not good at math but there exist so many > combinations that we will not be able to handle all of them. > n inputs, m encodings, l browsers, alteast 2 languages (J/V S), k > places where input is echoed, p weird things inside browsers.
> So conclusion is that even sandbox may not help much. Then why invest > so much into it. And why make PHPIDS heavier. > Really speaking I would like it to remain RegEx based. Thats the way I > love it.
> We should cover as many cases with regexes as possible I agree, but if > we can't promise a complete solution using SandBox then why bother > using it.
> -Kishor
> On Jul 5, 11:06 am, SirDarckCat <sirdarck...@gmail.com> wrote: > > Hi Mario.
> > About the php javascript implementation, I made a simple JavaScript > > interpreter in PHP (that uses spidermonkey) and makes the following:
> > This could be an example.. > > $real_code=evalJS("try{ return new > > Function(unescape('".rawurlencode($input_code)."')).toSource(); } > > catch(e) { return e.description; }");
> > That will return the "real code" of the function, and wont execute > > it.. the real code I mean, that it will delete useless parenthesis, > > useless white spaces, etc.. and that should make easier for checking > > with regular expressions.
> > Any way, I think that with some work the regular expressions will work > > as well, and wont need the client to have access to the passthru > > function.
> > About the:
> > > Also I am planning to write a rather detailed and in depth blog post about > > > advanced filter evasion for gnucitizen.org and I would like to use some of > > > the vectors as examples (the knowledge we gathered here has to be persisted) > > > - you're ok with that?
> > > Well - I thought about implementing a JavaScript runtime and discussed that > > > with Kishor via chat. But the problem is - if we'd implement that we'd also > > > have to implement one or several SQL sanddboxes to check on the SQL > > > injections, file system stubs to check on the directory traversal and so on. > > > Furthermore we'd loose the basic principle of the application - its light > > > weight.
> > > Also the PHPIDS is all about enumerating badness - and weighting and > > > assessing them. The only chance we have to deliver a value like the impact > > > is to keep a library of possibilities. Of course, the regular expressions > > > need optimizations here and there but I am still the opinion that this is > > > the right way. Imagine the situation we are using a JS interpreter and one > > > vector slips through - we'd need weeks to file the bug to the developers, > > > wait for patches, test ourselves etc. The rules are fixed and committed > > > within hours. Comparable thing with the halfwidth/fullwidth issue and the > > > hardware IDS systems - we were able to detect that input one hour after > > > being posted on sla.ckers/ in the sec news.
> > > Of course we have a kind of arms race at the moment to keep up with your > > > awesome findings but isn't that what it's all about? I've been developing > > > with JS for years and never learned that much about the basic fundamental > > > principles and - let's call them features - of this language. I luv it! I've > > > never seen XSS vectors that advanced like in the last three days when Sir > > > Darck Cat and you appeared in the group and I guess I am not alone with that > > > attitude.
> > > So my proposal is to keep the mentioned principles of the PHPIDS, work out > > > the current issues (has anyone studied math in here?) and keep enhancing the > > > software. Btw - it goes without saying that you guys will be credited in the > > > release notes for the coming release as well as on the contact/credits page > > > of php-ids.org ;)
> > > Also I am planning to write a rather detailed and in depth blog post about > > > advanced filter evasion for gnucitizen.org and I would like to use some of > > > the vectors as examples (the knowledge we gathered here has to be persisted) > > > - you're ok with that?
> > > > On 4 jul, 18:08, Giorgio Maone <giorgio.ma...@gmail.com> wrote: > > > > > On Jul 5, 12:00 am, ".mario" <Mario.Heider...@googlemail.com> wrote:
> > > > > > - various types of concatenation - maybe combined with charcode > > > > > > - detection of charcode built together by advanced arithmetic > > > > > > operations including parenthesis and floats
> > > > > At least, that's how I implemented NoScript "light" filters... a lot > > > > > less headaches, and most important, no "enumerating badness" > > > > > madness :)http://www.google.it/search?q=enumerating+badness
I understand what you say about the interpretter and the sandbox, I think you are right, the interpreter may requiere a ot of time.
Martin:
Hi! well, there is a trend, now to say that XSS is the new BoF, and JS the new shellcode. in that case, some IDS, try to detect shellcodes (real shellcodes), and that's why now exist's a lot of "shellcode encoders", this is possible in ASM, because the opcodes can be made with printable chars, but the case of javascript is different, the most filters mario puts, the harder is to find an attack vector, and even if we dont detect an approach, if someone does, I dont think it'll stay on the wild for a long time.. because the affected website will report that the IDS failed to detect the attack.
Greetings!!
On 5 jul, 08:58, "Martin Hinks" <mhi...@gmail.com> wrote:
> I think a complete runtime is absolute overkill and I think we are > already verging on too much interpretation of the JavaScript. I think > we should focus on detecting ways of executing a payload rather than > getting too hung up on the payload itself. In other words, what we can > detect with regex, we detect, what we cannot we just accept that it is > never going to cath everything without a runtime.
> For example: How can any of the JavaScript be used if it is impossible > to inject into a script tag or attribute? If we are catching attempts > to inject the payload in the first place and giving it a high impact > rating then it will be impossible to actually execute the payload > anyway and any other information we can glean about the payload is > merely an impact raiser.
> I too have concerns about the implementation of output scanning in > .NETIDS - it is *too* prone to false positives at the moment which is > why, in next weeks release, I am going to remove it by default but > also leave the capability in there as I think it is a useful tool - if > only for debugging purposes to see where there is potential for > fragmented injection. The output scanning works on a much smaller > ruleset than the main scanner but if you have any other suggestions > for how best to deal with this then I would like to hear it.
> Martin
> On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > According to me, there are several things to consider here.
> > 1) Implementing a JavaScript runtime: This idea may initially sound > > really cool, but I am of the opinion that it will not be very useful. > > There exist 10 different browsers and we need one for each of them. > > Also having a SQL sandbox is an overkill.
> > Let me first quote my interpretation of the use of sandbox here, > > if (thats correct) > > { > > please continue reading. > > } > > else > > { > > please explain what it is -again :) > > goto next_reply; > > }
> > Here it goes: "a) After IDS receives an input, we pass the input to > > the sandbox and determine if there is a valid JS code in the input. We > > also determine if there is any JS at all. > > b) Other alternative: Before the response is sent to the client, we > > pass it trhough the sandbox, determine the JS that potentially got > > added and either block the response or filter the new JS
> > "
> > If any of the above statements is true, the important thing to > > consider is how an input is mapped to the output? > > In case a), input many times is not going to be in valid JS syntax. > > E.g. if I am injecting inside a script tag, > > the input looks like "; alert(1); which isn't valid JS, JS interpreter > > will say there isn't a valid JS code.
> > In case b), we have no way to know which code was added by an > > application and which by attacker. We can't assume that attacker will > > always inject alerts and things to exploit/discover XSS. So looking > > inside the sandbox for evidence of alert etc. is not appropriate. (I > > have concerns with .NETIDS)
> > There can sometimes be conversions applied by the app on input which > > makes it vulnerable. In a hypothetical example if app converts 1 to > > 'on' and 0 to off, then I can inject 1mouseover and injection > > succeeds. We can't do much in these situations.
> > On math question, I am not good at math but there exist so many > > combinations that we will not be able to handle all of them. > > n inputs, m encodings, l browsers, alteast 2 languages (J/V S), k > > places where input is echoed, p weird things inside browsers.
> > So conclusion is that even sandbox may not help much. Then why invest > > so much into it. And why make PHPIDS heavier. > > Really speaking I would like it to remain RegEx based. Thats the way I > > love it.
> > We should cover as many cases with regexes as possible I agree, but if > > we can't promise a complete solution using SandBox then why bother > > using it.
> > -Kishor
> > On Jul 5, 11:06 am, SirDarckCat <sirdarck...@gmail.com> wrote: > > > Hi Mario.
> > > About the php javascript implementation, I made a simple JavaScript > > > interpreter in PHP (that uses spidermonkey) and makes the following:
> > > This could be an example.. > > > $real_code=evalJS("try{ return new > > > Function(unescape('".rawurlencode($input_code)."')).toSource(); } > > > catch(e) { return e.description; }");
> > > That will return the "real code" of the function, and wont execute > > > it.. the real code I mean, that it will delete useless parenthesis, > > > useless white spaces, etc.. and that should make easier for checking > > > with regular expressions.
> > > Any way, I think that with some work the regular expressions will work > > > as well, and wont need the client to have access to the passthru > > > function.
> > > About the:
> > > > Also I am planning to write a rather detailed and in depth blog post about > > > > advanced filter evasion for gnucitizen.org and I would like to use some of > > > > the vectors as examples (the knowledge we gathered here has to be persisted) > > > > - you're ok with that?
> > > > Well - I thought about implementing a JavaScript runtime and discussed that > > > > with Kishor via chat. But the problem is - if we'd implement that we'd also > > > > have to implement one or several SQL sanddboxes to check on the SQL > > > > injections, file system stubs to check on the directory traversal and so on. > > > > Furthermore we'd loose the basic principle of the application - its light > > > > weight.
> > > > Also the PHPIDS is all about enumerating badness - and weighting and > > > > assessing them. The only chance we have to deliver a value like the impact > > > > is to keep a library of possibilities. Of course, the regular expressions > > > > need optimizations here and there but I am still the opinion that this is > > > > the right way. Imagine the situation we are using a JS interpreter and one > > > > vector slips through - we'd need weeks to file the bug to the developers, > > > > wait for patches, test ourselves etc. The rules are fixed and committed > > > > within hours. Comparable thing with the halfwidth/fullwidth issue and the > > > > hardware IDS systems - we were able to detect that input one hour after > > > > being posted on sla.ckers/ in the sec news.
> > > > Of course we have a kind of arms race at the moment to keep up with your > > > > awesome findings but isn't that what it's all about? I've been developing > > > > with JS for years and never learned that much about the basic fundamental > > > > principles and - let's call them features - of this language. I luv it! I've > > > > never seen XSS vectors that advanced like in the last three days when Sir > > > > Darck Cat and you appeared in the group and I guess I am not alone with that > > > > attitude.
> > > > So my proposal is to keep the mentioned principles of the PHPIDS, work out > > > > the current issues (has anyone studied math in here?) and keep enhancing the > > > > software. Btw - it goes without saying that you guys will be credited in the > > > > release notes for the coming release as well as on the contact/credits page > > > > of php-ids.org ;)
> > > > Also I am planning to write a rather detailed and in depth blog post about > > > > advanced filter evasion for gnucitizen.org and I would like to use some of > > > > the vectors as examples (the knowledge we gathered here has to be persisted) > > > > - you're ok with that?
> > > > > On 4 jul, 18:08, Giorgio Maone <giorgio.ma...@gmail.com> wrote: > > > > > > On Jul 5, 12:00 am, ".mario" <Mario.Heider...@googlemail.com> wrote:
> > > > > > > - various types of concatenation - maybe combined with charcode > > > > > > > - detection of charcode built together by advanced arithmetic > > > > > > > operations including parenthesis and floats
> I think a complete runtime is absolute overkill and I think we are > already verging on too much interpretation of the JavaScript. I think > we should focus on detecting ways of executing a payload rather than > getting too hung up on the payload itself. In other words, what we can > detect with regex, we detect, what we cannot we just accept that it is > never going to cath everything without a runtime.
> For example: How can any of the JavaScript be used if it is impossible > to inject into a script tag or attribute? If we are catching attempts > to inject the payload in the first place and giving it a high impact > rating then it will be impossible to actually execute the payload > anyway and any other information we can glean about the payload is > merely an impact raiser.
> I too have concerns about the implementation of output scanning in > .NETIDS - it is *too* prone to false positives at the moment which is > why, in next weeks release, I am going to remove it by default but > also leave the capability in there as I think it is a useful tool - if > only for debugging purposes to see where there is potential for > fragmented injection. The output scanning works on a much smaller > ruleset than the main scanner but if you have any other suggestions > for how best to deal with this then I would like to hear it.
> Martin
> On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > According to me, there are several things to consider here.
> > 1) Implementing a JavaScript runtime: This idea may initially sound > > really cool, but I am of the opinion that it will not be very useful. > > There exist 10 different browsers and we need one for each of them. > > Also having a SQL sandbox is an overkill.
> > Let me first quote my interpretation of the use of sandbox here, > > if (thats correct) > > { > > please continue reading. > > } > > else > > { > > please explain what it is -again :) > > goto next_reply; > > }
> > Here it goes: "a) After IDS receives an input, we pass the input to > > the sandbox and determine if there is a valid JS code in the input. We > > also determine if there is any JS at all. > > b) Other alternative: Before the response is sent to the client, we > > pass it trhough the sandbox, determine the JS that potentially got > > added and either block the response or filter the new JS
> > "
> > If any of the above statements is true, the important thing to > > consider is how an input is mapped to the output? > > In case a), input many times is not going to be in valid JS syntax. > > E.g. if I am injecting inside a script tag, > > the input looks like "; alert(1); which isn't valid JS, JS interpreter > > will say there isn't a valid JS code.
> > In case b), we have no way to know which code was added by an > > application and which by attacker. We can't assume that attacker will > > always inject alerts and things to exploit/discover XSS. So looking > > inside the sandbox for evidence of alert etc. is not appropriate. (I > > have concerns with .NETIDS)
> > There can sometimes be conversions applied by the app on input which > > makes it vulnerable. In a hypothetical example if app converts 1 to > > 'on' and 0 to off, then I can inject 1mouseover and injection > > succeeds. We can't do much in these situations.
> > On math question, I am not good at math but there exist so many > > combinations that we will not be able to handle all of them. > > n inputs, m encodings, l browsers, alteast 2 languages (J/V S), k > > places where input is echoed, p weird things inside browsers.
> > So conclusion is that even sandbox may not help much. Then why invest > > so much into it. And why make PHPIDS heavier. > > Really speaking I would like it to remain RegEx based. Thats the way I > > love it.
> > We should cover as many cases with regexes as possible I agree, but if > > we can't promise a complete solution using SandBox then why bother > > using it.
> > -Kishor
> > On Jul 5, 11:06 am, SirDarckCat <sirdarck...@gmail.com> wrote: > > > Hi Mario.
> > > About the php javascript implementation, I made a simple JavaScript > > > interpreter in PHP (that uses spidermonkey) and makes the following:
> > > This could be an example.. > > > $real_code=evalJS("try{ return new > > > Function(unescape('".rawurlencode($input_code)."')).toSource(); } > > > catch(e) { return e.description; }");
> > > That will return the "real code" of the function, and wont execute > > > it.. the real code I mean, that it will delete useless parenthesis, > > > useless white spaces, etc.. and that should make easier for checking > > > with regular expressions.
> > > Any way, I think that with some work the regular expressions will work > > > as well, and wont need the client to have access to the passthru > > > function.
> > > About the:
> > > > Also I am planning to write a rather detailed and in depth blog post about > > > > advanced filter evasion for gnucitizen.org and I would like to use some of > > > > the vectors as examples (the knowledge we gathered here has to be persisted) > > > > - you're ok with that?
> > > > Well - I thought about implementing a JavaScript runtime and discussed that > > > > with Kishor via chat. But the problem is - if we'd implement that we'd also > > > > have to implement one or several SQL sanddboxes to check on the SQL > > > > injections, file system stubs to check on the directory traversal and so on. > > > > Furthermore we'd loose the basic principle of the application - its light > > > > weight.
> > > > Also the PHPIDS is all about enumerating badness - and weighting and > > > > assessing them. The only chance we have to deliver a value like the impact > > > > is to keep a library of possibilities. Of course, the regular expressions > > > > need optimizations here and there but I am still the opinion that this is > > > > the right way. Imagine the situation we are using a JS interpreter and one > > > > vector slips through - we'd need weeks to file the bug to the developers, > > > > wait for patches, test ourselves etc. The rules are fixed and committed > > > > within hours. Comparable thing with the halfwidth/fullwidth issue and the > > > > hardware IDS systems - we were able to detect that input one hour after > > > > being posted on sla.ckers/ in the sec news.
> > > > Of course we have a kind of arms race at the moment to keep up with your > > > > awesome findings but isn't that what it's all about? I've been developing > > > > with JS for years and never learned that much about the basic fundamental > > > > principles and - let's call them features - of this language. I luv it! I've > > > > never seen XSS vectors that advanced like in the last three days when Sir > > > > Darck Cat and you appeared in the group and I guess I am not alone with that > > > > attitude.
> > > > So my proposal is to keep the mentioned principles of the PHPIDS, work out > > > > the current issues (has anyone studied math in here?) and keep enhancing the > > > > software. Btw - it goes without saying that you guys will be credited in the > > > > release notes for the coming release as well as on the contact/credits page > > > > of php-ids.org ;)
> > > > Also I am planning to write a rather detailed and in depth blog post about > > > > advanced filter evasion for gnucitizen.org and I would like to use some of > > > > the vectors as examples (the knowledge we gathered here has to be persisted) > > > > - you're ok with that?
> > > > > On 4 jul, 18:08, Giorgio Maone <giorgio.ma...@gmail.com> wrote: > > > > > > On Jul 5, 12:00 am, ".mario" <Mario.Heider...@googlemail.com> wrote:
> > > > > > > - various types of concatenation - maybe combined with charcode > > > > > > > - detection of charcode built together by advanced arithmetic > > > > > > > operations including parenthesis and floats
Sadly I'm pretty sure that approach is not possible in .NET - I'd have to replace the Request object with a new version that implemented all members whilst overriding the GET and SET methods on each variable. But what does that actually achieve...? The problem that the Ouput Scanning is trying to deal with is concaternation of different querystrings/post parameters without permutation. ie:
If the user echoed out test + test2 + test4 (POST) + test3 the resulting output is <script>. To concat all of these in every possible way to look for fragmented injections would be too time consuming whereas if a user has a page in which they are writing raw data out and they want it to be output scanned they can compare the results of a legitimate request (ie. with no malicious input the page scores impact 10 for example) with a malicious request (if it scores 20 on a request then the actual impact is 20-10=10).
However, as I said, the problem this is trying to overcome is that each of those parameters on its own will throw no warning when put through the IDS - it is only when the user parses them and then writes them out to a control such as asp:Literal or asp:Hyperlink or even Page.Title, 3 of the very few controls that do not automatically encode output in .NET, that the problem exists. So, even if I were to control access to each querystring - without permuting I have no way of knowing how the fragmented XSS will be constructed.
You do realise that .NETIDS does all the same filtering as PHPIDS first on all incoming fields and Output Scanning is just an experimental bolt-on (at the moment)? Just wanted to make that clear hehe :p
If anyone is seeing anything I'm not here for detection of fragmented XSS then I'd really appreciate help - the permutations will just end up taking too long to compute whereas attaching an Output Filter is (relatively) trivial in .NET.
@SirDarckCat: Hi!
Been reading your posts but sitting quietly - they are great vectors - many thanks! In case you didn't realise I'm the culprit(!) for the .NET version of PHPIDS...
I agree that it is sensible to detect as much as is possible, but I think this should be within the bounds of Regular Expressions as far as we can - writing a JavaScript implementation is going to be VERY labour intensive - it will also have to emulate different browser behaviours - will undoubtedly be prone to infinite loops, be *slow* and in general be a nightmare to maintain as opposed to the regexen we currently employ. Detecting things like a=0||BLAH is a better approach than attempting to work out via interpretative logic that a=0||'e' + 'v' + 'a' + 'l' is a concatenation of eval. I'm not at any point arguing against detecting the payload, merely pointing out that to take this to the nth degree and implement a runtime seems overkill if there is no method for injecting these highly specific payloads in the first place.
Hope I made sense here and looking forward to other's feedback on the issues.
> On Jul 5, 3:58 pm, "Martin Hinks" <mhi...@gmail.com> wrote: > > @Kishord: great summary
> > I think a complete runtime is absolute overkill and I think we are > > already verging on too much interpretation of the JavaScript. I think > > we should focus on detecting ways of executing a payload rather than > > getting too hung up on the payload itself. In other words, what we can > > detect with regex, we detect, what we cannot we just accept that it is > > never going to cath everything without a runtime.
> > For example: How can any of the JavaScript be used if it is impossible > > to inject into a script tag or attribute? If we are catching attempts > > to inject the payload in the first place and giving it a high impact > > rating then it will be impossible to actually execute the payload > > anyway and any other information we can glean about the payload is > > merely an impact raiser.
> > I too have concerns about the implementation of output scanning in > > .NETIDS - it is *too* prone to false positives at the moment which is > > why, in next weeks release, I am going to remove it by default but > > also leave the capability in there as I think it is a useful tool - if > > only for debugging purposes to see where there is potential for > > fragmented injection. The output scanning works on a much smaller > > ruleset than the main scanner but if you have any other suggestions > > for how best to deal with this then I would like to hear it.
> > Martin
> > On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > > According to me, there are several things to consider here.
> > > 1) Implementing a JavaScript runtime: This idea may initially sound > > > really cool, but I am of the opinion that it will not be very useful. > > > There exist 10 different browsers and we need one for each of them. > > > Also having a SQL sandbox is an overkill.
> > > Let me first quote my interpretation of the use of sandbox here, > > > if (thats correct) > > > { > > > please continue reading. > > > } > > > else > > > { > > > please explain what it is -again :) > > > goto next_reply; > > > }
> > > Here it goes: "a) After IDS receives an input, we pass the input to > > > the sandbox and determine if there is a valid JS code in the input. We > > > also determine if there is any JS at all. > > > b) Other alternative: Before the response is sent to the client, we > > > pass it trhough the sandbox, determine the JS that potentially got > > > added and either block the response or filter the new JS
> > > "
> > > If any of the above statements is true, the important thing to > > > consider is how an input is mapped to the output? > > > In case a), input many times is not going to be in valid JS syntax. > > > E.g. if I am injecting inside a script tag, > > > the input looks like "; alert(1); which isn't valid JS, JS interpreter > > > will say there isn't a valid JS code.
> > > In case b), we have no way to know which code was added by an > > > application and which by attacker. We can't assume that attacker will > > > always inject alerts and things to exploit/discover XSS. So looking > > > inside the sandbox for evidence of alert etc. is not appropriate. (I > > > have concerns with .NETIDS)
> > > There can sometimes be conversions applied by the app on input which > > > makes it vulnerable. In a hypothetical example if app converts 1 to > > > 'on' and 0 to off, then I can inject 1mouseover and injection > > > succeeds. We can't do much in these situations.
> > > On math question, I am not good at math but there exist so many > > > combinations that we will not be able to handle all of them. > > > n inputs, m encodings, l browsers, alteast 2 languages (J/V S), k > > > places where input is echoed, p weird things inside browsers.
> > > So conclusion is that even sandbox may not help much. Then why invest > > > so much into it. And why make PHPIDS heavier. > > > Really speaking I would like it to remain RegEx based. Thats the way I > > > love it.
> > > We should cover as many cases with regexes as possible I agree, but if > > > we can't promise a complete solution using SandBox then why bother > > > using it.
> > > -Kishor
> > > On Jul 5, 11:06 am, SirDarckCat <sirdarck...@gmail.com> wrote: > > > > Hi Mario.
> > > > About the php javascript implementation, I made a simple JavaScript > > > > interpreter in PHP (that uses spidermonkey) and makes the following:
> > > > This could be an example.. > > > > $real_code=evalJS("try{ return new > > > > Function(unescape('".rawurlencode($input_code)."')).toSource(); } > > > > catch(e) { return e.description; }");
> > > > That will return the "real code" of the function, and wont execute > > > > it.. the real code I mean, that it will delete useless parenthesis, > > > > useless white spaces, etc.. and that should make easier for checking > > > > with regular expressions.
> > > > Any way, I think that with some work the regular expressions will work > > > > as well, and wont need the client to have access to the passthru > > > > function.
> > > > About the:
> > > > > Also I am planning to write a rather detailed and in depth blog post about > > > > > advanced filter evasion for gnucitizen.org and I would like to use some of > > > > > the vectors as examples (the knowledge we gathered here has to be persisted) > > > > > - you're ok with that?
> Sadly I'm pretty sure that approach is not possible in .NET - I'd have > to replace the Request object with a new version that implemented all > members whilst overriding the GET and SET methods on each variable. > But what does that actually achieve...? The problem that the Ouput > Scanning is trying to deal with is concaternation of different > querystrings/post parameters without permutation. ie:
> If the user echoed out test + test2 + test4 (POST) + test3 the > resulting output is <script>. To concat all of these in every possible > way to look for fragmented injections would be too time consuming > whereas if a user has a page in which they are writing raw data out > and they want it to be output scanned they can compare the results of > a legitimate request (ie. with no malicious input the page scores > impact 10 for example) with a malicious request (if it scores 20 on a > request then the actual impact is 20-10=10).
> However, as I said, the problem this is trying to overcome is that > each of those parameters on its own will throw no warning when put > through the IDS - it is only when the user parses them and then writes > them out to a control such as asp:Literal or asp:Hyperlink or even > Page.Title, 3 of the very few controls that do not automatically > encode output in .NET, that the problem exists. So, even if I were to > control access to each querystring - without permuting I have no way > of knowing how the fragmented XSS will be constructed.
> You do realise that .NETIDS does all the same filtering as PHPIDS > first on all incoming fields and Output Scanning is just an > experimental bolt-on (at the moment)? Just wanted to make that clear > hehe :p
> If anyone is seeing anything I'm not here for detection of fragmented > XSS then I'd really appreciate help - the permutations will just end > up taking too long to compute whereas attaching an Output Filter is > (relatively) trivial in .NET.
> @SirDarckCat: Hi!
> Been reading your posts but sitting quietly - they are great vectors - > many thanks! In case you didn't realise I'm the culprit(!) for the > .NET version of PHPIDS...
> I agree that it is sensible to detect as much as is possible, but I > think this should be within the bounds of Regular Expressions as far > as we can - writing a JavaScript implementation is going to be VERY > labour intensive - it will also have to emulate different browser > behaviours - will undoubtedly be prone to infinite loops, be *slow* > and in general be a nightmare to maintain as opposed to the regexen we > currently employ. Detecting things like a=0||BLAH is a better approach > than attempting to work out via interpretative logic that a=0||'e' + > 'v' + 'a' + 'l' is a concatenation of eval. I'm not at any point > arguing against detecting the payload, merely pointing out that to > take this to the nth degree and implement a runtime seems overkill if > there is no method for injecting these highly specific payloads in the > first place.
> Hope I made sense here and looking forward to other's feedback on the issues.
> Martin
> On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > Hi Martin,
> > I am not a .NET expert, but if .NETIDS can add handlers as getters of > > the query parameter elements, maybe it will be a little better.
> > So essentially you apply output filter not to the response but to the > > query elements that are 'echoed'.
> > On Jul 5, 3:58 pm, "Martin Hinks" <mhi...@gmail.com> wrote: > > > @Kishord: great summary
> > > I think a complete runtime is absolute overkill and I think we are > > > already verging on too much interpretation of the JavaScript. I think > > > we should focus on detecting ways of executing a payload rather than > > > getting too hung up on the payload itself. In other words, what we can > > > detect with regex, we detect, what we cannot we just accept that it is > > > never going to cath everything without a runtime.
> > > For example: How can any of the JavaScript be used if it is impossible > > > to inject into a script tag or attribute? If we are catching attempts > > > to inject the payload in the first place and giving it a high impact > > > rating then it will be impossible to actually execute the payload > > > anyway and any other information we can glean about the payload is > > > merely an impact raiser.
> > > I too have concerns about the implementation of output scanning in > > > .NETIDS - it is *too* prone to false positives at the moment which is > > > why, in next weeks release, I am going to remove it by default but > > > also leave the capability in there as I think it is a useful tool - if > > > only for debugging purposes to see where there is potential for > > > fragmented injection. The output scanning works on a much smaller > > > ruleset than the main scanner but if you have any other suggestions > > > for how best to deal with this then I would like to hear it.
> > > Martin
> > > On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > > > According to me, there are several things to consider here.
> > > > 1) Implementing a JavaScript runtime: This idea may initially sound > > > > really cool, but I am of the opinion that it will not be very useful. > > > > There exist 10 different browsers and we need one for each of them. > > > > Also having a SQL sandbox is an overkill.
> > > > Let me first quote my interpretation of the use of sandbox here, > > > > if (thats correct) > > > > { > > > > please continue reading. > > > > } > > > > else > > > > { > > > > please explain what it is -again :) > > > > goto next_reply; > > > > }
> > > > Here it goes: "a) After IDS receives an input, we pass the input to > > > > the sandbox and determine if there is a valid JS code in the input. We > > > > also determine if there is any JS at all. > > > > b) Other alternative: Before the response is sent to the client, we > > > > pass it trhough the sandbox, determine the JS that potentially got > > > > added and either block the response or filter the new JS
> > > > "
> > > > If any of the above statements is true, the important thing to > > > > consider is how an input is mapped to the output? > > > > In case a), input many times is not going to be in valid JS syntax. > > > > E.g. if I am injecting inside a script tag, > > > > the input looks like "; alert(1); which isn't valid JS, JS interpreter > > > > will say there isn't a valid JS code.
> > > > In case b), we have no way to know which code was added by an > > > > application and which by attacker. We can't assume that attacker will > > > > always inject alerts and things to exploit/discover XSS. So looking > > > > inside the sandbox for evidence of alert etc. is not appropriate. (I > > > > have concerns with .NETIDS)
> > > > There can sometimes be conversions applied by the app on input which > > > > makes it vulnerable. In a hypothetical example if app converts 1 to > > > > 'on' and 0 to off, then I can inject 1mouseover and injection > > > > succeeds. We can't do much in these situations.
> > > > On math question, I am not good at math but there exist so many > > > > combinations that we will not be able to handle all of them. > > > > n inputs, m encodings, l browsers, alteast 2 languages (J/V S), k > > > > places where input is echoed, p weird things inside browsers.
> > > > So conclusion is that even sandbox may not help much. Then why invest > > > > so much into it. And why make PHPIDS heavier. > > > > Really speaking I would like it to remain RegEx based. Thats the way I > > > > love it.
> > > > We should cover as many cases with regexes as possible I agree, but if > > > > we can't promise a complete solution using SandBox then why bother > > > > using it.
> > > > -Kishor
> > > > On Jul 5, 11:06 am, SirDarckCat <sirdarck...@gmail.com> wrote: > > > > > Hi Mario.
> > > > > About the php javascript implementation, I made a simple JavaScript > > > > > interpreter in PHP (that uses spidermonkey) and makes the following:
> > > > > This could be an example.. > > > > > $real_code=evalJS("try{ return new > > > > > Function(unescape('".rawurlencode($input_code)."')).toSource(); } > > > > > catch(e) { return e.description; }");
> > > > > That will return the "real code" of the function, and wont execute > > > > > it.. the real code I mean, that it will delete useless parenthesis, > > > > > useless white spaces, etc.. and that should make easier for checking > > > > > with regular expressions.
> > > > > Any way, I think that with some work the regular expressions will work > > > > > as well, and wont need the client to have access to the passthru > > > > > function.
Finally I come to reading your posts and thinking about them - just fixed the converter class and optimized some rules.
I agree with Martin and Kishor. Yes - a JS engine would be definitely very cool but too overweighted and question the concept of the PHPIDS. Also the charcode conversion I meanwhile think is a nice to have in a basic variant but since it's impossible to cover all different ways to calculate the charcode together it will always be possible to circumvent the IDS converters.
Fragmented XSS is definitely an issue very hard to come by - at least with PHP. I envy .NET a little bit for just being more strict and structured and having better possibilities to deal with that one. With PHP there's no real chance to make an application that covers any way of rendering output and have comparison mechanisms of the should' and the 'is'. On the other hand fragmented XSS is still found pretty seldom (or am I wrong?) and it would be another 'nice to have' like the advanced charcode stuff.
What do you think? Greetings, .mario
On Jul 5, 6:13 pm, "Martin Hinks" <mhi...@gmail.com> wrote:
> Sadly I'm pretty sure that approach is not possible in .NET - I'd have > to replace the Request object with a new version that implemented all > members whilst overriding the GET and SET methods on each variable. > But what does that actually achieve...? The problem that the Ouput > Scanning is trying to deal with is concaternation of different > querystrings/post parameters without permutation. ie:
> If the user echoed out test + test2 + test4 (POST) + test3 the > resulting output is <script>. To concat all of these in every possible > way to look for fragmented injections would be too time consuming > whereas if a user has a page in which they are writing raw data out > and they want it to be output scanned they can compare the results of > a legitimate request (ie. with no malicious input the page scores > impact 10 for example) with a malicious request (if it scores 20 on a > request then the actual impact is 20-10=10).
> However, as I said, the problem this is trying to overcome is that > each of those parameters on its own will throw no warning when put > through the IDS - it is only when the user parses them and then writes > them out to a control such as asp:Literal or asp:Hyperlink or even > Page.Title, 3 of the very few controls that do not automatically > encode output in .NET, that the problem exists. So, even if I were to > control access to each querystring - without permuting I have no way > of knowing how the fragmented XSS will be constructed.
> You do realise that .NETIDS does all the same filtering as PHPIDS > first on all incoming fields and Output Scanning is just an > experimental bolt-on (at the moment)? Just wanted to make that clear > hehe :p
> If anyone is seeing anything I'm not here for detection of fragmented > XSS then I'd really appreciate help - the permutations will just end > up taking too long to compute whereas attaching an Output Filter is > (relatively) trivial in .NET.
> @SirDarckCat: Hi!
> Been reading your posts but sitting quietly - they are great vectors - > many thanks! In case you didn't realise I'm the culprit(!) for the > .NET version of PHPIDS...
> I agree that it is sensible to detect as much as is possible, but I > think this should be within the bounds of Regular Expressions as far > as we can - writing a JavaScript implementation is going to be VERY > labour intensive - it will also have to emulate different browser > behaviours - will undoubtedly be prone to infinite loops, be *slow* > and in general be a nightmare to maintain as opposed to the regexen we > currently employ. Detecting things like a=0||BLAH is a better approach > than attempting to work out via interpretative logic that a=0||'e' + > 'v' + 'a' + 'l' is a concatenation of eval. I'm not at any point > arguing against detecting the payload, merely pointing out that to > take this to the nth degree and implement a runtime seems overkill if > there is no method for injecting these highly specific payloads in the > first place.
> Hope I made sense here and looking forward to other's feedback on the issues.
> Martin
> On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > Hi Martin,
> > I am not a .NET expert, but if .NETIDS can add handlers as getters of > > the query parameter elements, maybe it will be a little better.
> > So essentially you apply output filter not to the response but to the > > query elements that are 'echoed'.
> > On Jul 5, 3:58 pm, "Martin Hinks" <mhi...@gmail.com> wrote: > > > @Kishord: great summary
> > > I think a complete runtime is absolute overkill and I think we are > > > already verging on too much interpretation of the JavaScript. I think > > > we should focus on detecting ways of executing a payload rather than > > > getting too hung up on the payload itself. In other words, what we can > > > detect with regex, we detect, what we cannot we just accept that it is > > > never going to cath everything without a runtime.
> > > For example: How can any of the JavaScript be used if it is impossible > > > to inject into a script tag or attribute? If we are catching attempts > > > to inject the payload in the first place and giving it a high impact > > > rating then it will be impossible to actually execute the payload > > > anyway and any other information we can glean about the payload is > > > merely an impact raiser.
> > > I too have concerns about the implementation of output scanning in > > > .NETIDS - it is *too* prone to false positives at the moment which is > > > why, in next weeks release, I am going to remove it by default but > > > also leave the capability in there as I think it is a useful tool - if > > > only for debugging purposes to see where there is potential for > > > fragmented injection. The output scanning works on a much smaller > > > ruleset than the main scanner but if you have any other suggestions > > > for how best to deal with this then I would like to hear it.
> > > Martin
> > > On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > > > According to me, there are several things to consider here.
> > > > 1) Implementing a JavaScript runtime: This idea may initially sound > > > > really cool, but I am of the opinion that it will not be very useful. > > > > There exist 10 different browsers and we need one for each of them. > > > > Also having a SQL sandbox is an overkill.
> > > > Let me first quote my interpretation of the use of sandbox here, > > > > if (thats correct) > > > > { > > > > please continue reading. > > > > } > > > > else > > > > { > > > > please explain what it is -again :) > > > > goto next_reply; > > > > }
> > > > Here it goes: "a) After IDS receives an input, we pass the input to > > > > the sandbox and determine if there is a valid JS code in the input. We > > > > also determine if there is any JS at all. > > > > b) Other alternative: Before the response is sent to the client, we > > > > pass it trhough the sandbox, determine the JS that potentially got > > > > added and either block the response or filter the new JS
> > > > "
> > > > If any of the above statements is true, the important thing to > > > > consider is how an input is mapped to the output? > > > > In case a), input many times is not going to be in valid JS syntax. > > > > E.g. if I am injecting inside a script tag, > > > > the input looks like "; alert(1); which isn't valid JS, JS interpreter > > > > will say there isn't a valid JS code.
> > > > In case b), we have no way to know which code was added by an > > > > application and which by attacker. We can't assume that attacker will > > > > always inject alerts and things to exploit/discover XSS. So looking > > > > inside the sandbox for evidence of alert etc. is not appropriate. (I > > > > have concerns with .NETIDS)
> > > > There can sometimes be conversions applied by the app on input which > > > > makes it vulnerable. In a hypothetical example if app converts 1 to > > > > 'on' and 0 to off, then I can inject 1mouseover and injection > > > > succeeds. We can't do much in these situations.
> > > > On math question, I am not good at math but there exist so many > > > > combinations that we will not be able to handle all of them. > > > > n inputs, m encodings, l browsers, alteast 2 languages (J/V S), k > > > > places where input is echoed, p weird things inside browsers.
> > > > So conclusion is that even sandbox may not help much. Then why invest > > > > so much into it. And why make PHPIDS heavier. > > > > Really speaking I would like it to remain RegEx based. Thats the way I > > > > love it.
> > > > We should cover as many cases with regexes as possible I agree, but if > > > > we can't promise a complete solution using SandBox then why bother > > > > using it.
> > > > -Kishor
> > > > On Jul 5, 11:06 am, SirDarckCat <sirdarck...@gmail.com> wrote: > > > > > Hi Mario.
> > > > > About the php javascript implementation, I made a simple JavaScript > > > > > interpreter in PHP (that uses spidermonkey) and makes the following:
> Finally I come to reading your posts and thinking about them - just > fixed the converter class and optimized some rules.
> I agree with Martin and Kishor. Yes - a JS engine would be definitely > very cool but too overweighted and question the concept of the PHPIDS. > Also the charcode conversion I meanwhile think is a nice to have in a > basic variant but since it's impossible to cover all different ways to > calculate the charcode together it will always be possible to > circumvent the IDS converters.
> Fragmented XSS is definitely an issue very hard to come by - at least > with PHP. I envy .NET a little bit for just being more strict and > structured and having better possibilities to deal with that one. With > PHP there's no real chance to make an application that covers any way > of rendering output and have comparison mechanisms of the should' and > the 'is'. On the other hand fragmented XSS is still found pretty > seldom (or am I wrong?) and it would be another 'nice to have' like > the advanced charcode stuff.
> > Sadly I'm pretty sure that approach is not possible in .NET - I'd have > > to replace the Request object with a new version that implemented all > > members whilst overriding the GET and SET methods on each variable. > > But what does that actually achieve...? The problem that the Ouput > > Scanning is trying to deal with is concaternation of different > > querystrings/post parameters without permutation. ie:
> > If the user echoed out test + test2 + test4 (POST) + test3 the > > resulting output is <script>. To concat all of these in every possible > > way to look for fragmented injections would be too time consuming > > whereas if a user has a page in which they are writing raw data out > > and they want it to be output scanned they can compare the results of > > a legitimate request (ie. with no malicious input the page scores > > impact 10 for example) with a malicious request (if it scores 20 on a > > request then the actual impact is 20-10=10).
> > However, as I said, the problem this is trying to overcome is that > > each of those parameters on its own will throw no warning when put > > through the IDS - it is only when the user parses them and then writes > > them out to a control such as asp:Literal or asp:Hyperlink or even > > Page.Title, 3 of the very few controls that do not automatically > > encode output in .NET, that the problem exists. So, even if I were to > > control access to each querystring - without permuting I have no way > > of knowing how the fragmented XSS will be constructed.
> > You do realise that .NETIDS does all the same filtering as PHPIDS > > first on all incoming fields and Output Scanning is just an > > experimental bolt-on (at the moment)? Just wanted to make that clear > > hehe :p
> > If anyone is seeing anything I'm not here for detection of fragmented > > XSS then I'd really appreciate help - the permutations will just end > > up taking too long to compute whereas attaching an Output Filter is > > (relatively) trivial in .NET.
> > @SirDarckCat: Hi!
> > Been reading your posts but sitting quietly - they are great vectors - > > many thanks! In case you didn't realise I'm the culprit(!) for the > > .NET version of PHPIDS...
> > I agree that it is sensible to detect as much as is possible, but I > > think this should be within the bounds of Regular Expressions as far > > as we can - writing a JavaScript implementation is going to be VERY > > labour intensive - it will also have to emulate different browser > > behaviours - will undoubtedly be prone to infinite loops, be *slow* > > and in general be a nightmare to maintain as opposed to the regexen we > > currently employ. Detecting things like a=0||BLAH is a better approach > > than attempting to work out via interpretative logic that a=0||'e' + > > 'v' + 'a' + 'l' is a concatenation of eval. I'm not at any point > > arguing against detecting the payload, merely pointing out that to > > take this to the nth degree and implement a runtime seems overkill if > > there is no method for injecting these highly specific payloads in the > > first place.
> > Hope I made sense here and looking forward to other's feedback on the issues.
> > Martin
> > On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > > Hi Martin,
> > > I am not a .NET expert, but if .NETIDS can add handlers as getters of > > > the query parameter elements, maybe it will be a little better.
> > > So essentially you apply output filter not to the response but to the > > > query elements that are 'echoed'.
> > > On Jul 5, 3:58 pm, "Martin Hinks" <mhi...@gmail.com> wrote: > > > > @Kishord: great summary
> > > > I think a complete runtime is absolute overkill and I think we are > > > > already verging on too much interpretation of the JavaScript. I think > > > > we should focus on detecting ways of executing a payload rather than > > > > getting too hung up on the payload itself. In other words, what we can > > > > detect with regex, we detect, what we cannot we just accept that it is > > > > never going to cath everything without a runtime.
> > > > For example: How can any of the JavaScript be used if it is impossible > > > > to inject into a script tag or attribute? If we are catching attempts > > > > to inject the payload in the first place and giving it a high impact > > > > rating then it will be impossible to actually execute the payload > > > > anyway and any other information we can glean about the payload is > > > > merely an impact raiser.
> > > > I too have concerns about the implementation of output scanning in > > > > .NETIDS - it is *too* prone to false positives at the moment which is > > > > why, in next weeks release, I am going to remove it by default but > > > > also leave the capability in there as I think it is a useful tool - if > > > > only for debugging purposes to see where there is potential for > > > > fragmented injection. The output scanning works on a much smaller > > > > ruleset than the main scanner but if you have any other suggestions > > > > for how best to deal with this then I would like to hear it.
> > > > Martin
> > > > On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > > > > According to me, there are several things to consider here.
> > > > > 1) Implementing a JavaScript runtime: This idea may initially sound > > > > > really cool, but I am of the opinion that it will not be very useful. > > > > > There exist 10 different browsers and we need one for each of them. > > > > > Also having a SQL sandbox is an overkill.
> > > > > Let me first quote my interpretation of the use of sandbox here, > > > > > if (thats correct) > > > > > { > > > > > please continue reading. > > > > > } > > > > > else > > > > > { > > > > > please explain what it is -again :) > > > > > goto next_reply; > > > > > }
> > > > > Here it goes: "a) After IDS receives an input, we pass the input to > > > > > the sandbox and determine if there is a valid JS code in the input. We > > > > > also determine if there is any JS at all. > > > > > b) Other alternative: Before the response is sent to the client, we > > > > > pass it trhough the sandbox, determine the JS that potentially got > > > > > added and either block the response or filter the new JS
> > > > > "
> > > > > If any of the above statements is true, the important thing to > > > > > consider is how an input is mapped to the output? > > > > > In case a), input many times is not going to be in valid JS syntax. > > > > > E.g. if I am injecting inside a script tag, > > > > > the input looks like "; alert(1); which isn't valid JS, JS interpreter > > > > > will say there isn't a valid JS code.
> > > > > In case b), we have no way to know which code was added by an > > > > > application and which by attacker. We can't assume that attacker will > > > > > always inject alerts and things to exploit/discover XSS. So looking > > > > > inside the sandbox for evidence of alert etc. is not appropriate. (I > > > > > have concerns with .NETIDS)
> > > > > There can sometimes be conversions applied by the app on input which > > > > > makes it vulnerable. In a hypothetical example if app converts 1 to > > > > > 'on' and 0 to off, then I can inject 1mouseover and injection > > > > > succeeds. We can't do much in these situations.
> > > > > On math question, I am not good at math but there exist so many > > > > > combinations that we will not be able to handle all of them. > > > > > n inputs, m encodings, l browsers, alteast 2 languages (J/V S), k > > > > > places where input is echoed, p weird things inside browsers.
> > > > > So conclusion is that even sandbox may not help
Thanx - Awesome one Martin! This one definitely scratches the the sky. "reverse-concatenated hash-contained non-alpha-non-digit-obfuscated XSS" or what should we call that one?
> On 7/5/07, .mario <Mario.Heider...@googlemail.com> wrote:
> > Hi!
> > Finally I come to reading your posts and thinking about them - just > > fixed the converter class and optimized some rules.
> > I agree with Martin and Kishor. Yes - a JS engine would be definitely > > very cool but too overweighted and question the concept of the PHPIDS. > > Also the charcode conversion I meanwhile think is a nice to have in a > > basic variant but since it's impossible to cover all different ways to > > calculate the charcode together it will always be possible to > > circumvent the IDS converters.
> > Fragmented XSS is definitely an issue very hard to come by - at least > > with PHP. I envy .NET a little bit for just being more strict and > > structured and having better possibilities to deal with that one. With > > PHP there's no real chance to make an application that covers any way > > of rendering output and have comparison mechanisms of the should' and > > the 'is'. On the other hand fragmented XSS is still found pretty > > seldom (or am I wrong?) and it would be another 'nice to have' like > > the advanced charcode stuff.
> > > Sadly I'm pretty sure that approach is not possible in .NET - I'd have > > > to replace the Request object with a new version that implemented all > > > members whilst overriding the GET and SET methods on each variable. > > > But what does that actually achieve...? The problem that the Ouput > > > Scanning is trying to deal with is concaternation of different > > > querystrings/post parameters without permutation. ie:
> > > If the user echoed out test + test2 + test4 (POST) + test3 the > > > resulting output is <script>. To concat all of these in every possible > > > way to look for fragmented injections would be too time consuming > > > whereas if a user has a page in which they are writing raw data out > > > and they want it to be output scanned they can compare the results of > > > a legitimate request (ie. with no malicious input the page scores > > > impact 10 for example) with a malicious request (if it scores 20 on a > > > request then the actual impact is 20-10=10).
> > > However, as I said, the problem this is trying to overcome is that > > > each of those parameters on its own will throw no warning when put > > > through the IDS - it is only when the user parses them and then writes > > > them out to a control such as asp:Literal or asp:Hyperlink or even > > > Page.Title, 3 of the very few controls that do not automatically > > > encode output in .NET, that the problem exists. So, even if I were to > > > control access to each querystring - without permuting I have no way > > > of knowing how the fragmented XSS will be constructed.
> > > You do realise that .NETIDS does all the same filtering as PHPIDS > > > first on all incoming fields and Output Scanning is just an > > > experimental bolt-on (at the moment)? Just wanted to make that clear > > > hehe :p
> > > If anyone is seeing anything I'm not here for detection of fragmented > > > XSS then I'd really appreciate help - the permutations will just end > > > up taking too long to compute whereas attaching an Output Filter is > > > (relatively) trivial in .NET.
> > > @SirDarckCat: Hi!
> > > Been reading your posts but sitting quietly - they are great vectors - > > > many thanks! In case you didn't realise I'm the culprit(!) for the > > > .NET version of PHPIDS...
> > > I agree that it is sensible to detect as much as is possible, but I > > > think this should be within the bounds of Regular Expressions as far > > > as we can - writing a JavaScript implementation is going to be VERY > > > labour intensive - it will also have to emulate different browser > > > behaviours - will undoubtedly be prone to infinite loops, be *slow* > > > and in general be a nightmare to maintain as opposed to the regexen we > > > currently employ. Detecting things like a=0||BLAH is a better approach > > > than attempting to work out via interpretative logic that a=0||'e' + > > > 'v' + 'a' + 'l' is a concatenation of eval. I'm not at any point > > > arguing against detecting the payload, merely pointing out that to > > > take this to the nth degree and implement a runtime seems overkill if > > > there is no method for injecting these highly specific payloads in the > > > first place.
> > > Hope I made sense here and looking forward to other's feedback on the > issues.
> > > Martin
> > > On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > > > Hi Martin,
> > > > I am not a .NET expert, but if .NETIDS can add handlers as getters > of > > > > the query parameter elements, maybe it will be a little better.
> > > > So essentially you apply output filter not to the response but to > the > > > > query elements that are 'echoed'.
> > > > > I think a complete runtime is absolute overkill and I think we are > > > > > already verging on too much interpretation of the JavaScript. I > think > > > > > we should focus on detecting ways of executing a payload rather > than > > > > > getting too hung up on the payload itself. In other words, what we > can > > > > > detect with regex, we detect, what we cannot we just accept that > it is > > > > > never going to cath everything without a runtime.
> > > > > For example: How can any of the JavaScript be used if it is > impossible > > > > > to inject into a script tag or attribute? If we are catching > attempts > > > > > to inject the payload in the first place and giving it a high > impact > > > > > rating then it will be impossible to actually execute the payload > > > > > anyway and any other information we can glean about the payload is > > > > > merely an impact raiser.
> > > > > I too have concerns about the implementation of output scanning in > > > > > .NETIDS - it is *too* prone to false positives at the moment which > is > > > > > why, in next weeks release, I am going to remove it by default but > > > > > also leave the capability in there as I think it is a useful tool > - if > > > > > only for debugging purposes to see where there is potential for > > > > > fragmented injection. The output scanning works on a much smaller > > > > > ruleset than the main scanner but if you have any other > suggestions > > > > > for how best to deal with this then I would like to hear it.
> > > > > Martin
> > > > > On 7/5/07, kishord <kishor.t...@gmail.com> wrote:
> > > > > > According to me, there are several things to consider here.
> > > > > > 1) Implementing a JavaScript runtime: This idea may initially > sound > > > > > > really cool, but I am of the opinion that it will not be very > useful. > > > > > > There exist 10 different browsers and we need one for each of > them. > > > > > > Also having a SQL sandbox is an overkill.
> > > > > > Let me first quote my interpretation of the use of sandbox here, > > > > > > if (thats correct) > > > > > > { > > > > > > please continue reading. > > > > > > } > > > > > > else > > > > > > { > > > > > > please explain what it is -again :) > > > > > > goto next_reply; > > > > > > }
> > > > > > Here it goes: "a) After IDS receives an input, we pass the input > to > > > > > > the sandbox and determine if there is a valid JS code in the > input. We > > > > > > also determine if there is any JS at all. > > > > > > b) Other alternative: Before the response is sent to the client, > we > > > > > > pass it trhough the sandbox, determine the JS that potentially > got > > > > > > added and either block the response or filter the new JS
> > > > > > "
> > > > > > If any of the above statements is true, the important thing to > > > > > > consider is how an input is mapped to the output? > > > > > > In case a), input many times is not going to be in valid JS > syntax. > > > > > > E.g. if I am injecting inside a script tag, > > > > > > the input looks like "; alert(1); which isn't valid JS, JS > interpreter > > > > > > will say there isn't a valid JS code.
> > > > > > In case b), we have no way to know which code was added by an > > > > > > application and which by attacker. We can't assume that attacker > will > > > > > > always inject alerts and things to exploit/discover XSS. So > looking > > > > > > inside the sandbox
Regarding the JS interpreter VS enumerating badness discussion, it's worth noticing that I didn't have to adjust NoScript's Anti-XSS filters to catch any of these "new" vectors. I know it's a bit unfair because I "live" inside a browser, thus I don't need extra magic to interface SpiderMonkey, but as I said following this trend you'll end to rewrite a JS tokenizer using RegExps (which may even be an interesting project, BTW)
Actually I'm just optimizing NoScript's InjectionChecker for speed and using the extra wisdom acquired in this thread to *release* constraints :) I'm between 0 and 30 millisecs for complex URLs. Have you got any benchmark for (PHP|NET)IDS? -- There's a browser safer than Firefox... http://noscript.net
"I know it's a bit unfair because I "live" inside a browser" Yep - and damn I envy you a little bit ;)
We profile the PHPIDS with xdebug and as far as i can say it doesn't really hurt according to performance. The major bottlenecks weren't to find in the matching process it self.
A regex based JS parser is a pretty bizarre idea that the berlios guys already built - but the project is not more like a PoC in my opinion.
Greetings, .mario
On 6 Jul., 12:40, Giorgio Maone <giorgio.ma...@gmail.com> wrote:
> Regarding the JS interpreter VS enumerating badness discussion, it's > worth noticing that I didn't have to adjust NoScript's Anti-XSS > filters to catch any of these "new" vectors. > I know it's a bit unfair because I "live" inside a browser, thus I > don't need extra magic to interface SpiderMonkey, but as I said > following this trend you'll end to rewrite a JS tokenizer using > RegExps (which may even be an interesting project, BTW)
> Actually I'm just optimizing NoScript's InjectionChecker for speed and > using the extra wisdom acquired in this thread to *release* > constraints :) > I'm between 0 and 30 millisecs for complex URLs. Have you got any > benchmark for (PHP|NET)IDS? > -- > There's a browser safer than Firefox...http://noscript.net
|
