StackOverflowError while parsing long ACI with repetitive targets

[myandrews]

Describe the bug
We use OpenDJ 5.1.0 and one of our fuzzing test is failed for decoding long ACI with repetitive targets in it.

The problem ACI is - (targetscope="&O")(targetscope="&O")(targetscope="&O")...here comes hundreds of repetitive (targetscope="&O")(version 3.0; acl ""; allow (G'$z�?) ; )

The error is:
Caused by: java.lang.StackOverflowError
at java.base/java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:4134)
at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:5000)
at java.base/java.util.regex.Pattern$CharPropertyGreedy.match(Pattern.java:4470)
at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4969)
at java.base/java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:4134)
at java.base/java.util.regex.Pattern$BmpCharPropertyGreedy.match(Pattern.java:4509)
at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:5000)
at java.base/java.util.regex.Pattern$BmpCharProperty.match(Pattern.java:4134)
at java.base/java.util.regex.Pattern$Ques.match(Pattern.java:4410)
at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4969)
at java.base/java.util.regex.Pattern$BmpCharPropertyGreedy.match(Pattern.java:4509)
at java.base/java.util.regex.Pattern$GroupTail.match(Pattern.java:5000)
at java.base/java.util.regex.Pattern$BmpCharPropertyGreedy.match(Pattern.java:4509)
at java.base/java.util.regex.Pattern$GroupHead.match(Pattern.java:4969)
at java.base/java.util.regex.Pattern$BmpCharPropertyGreedy.match(Pattern.java:4509)

This happens due to Java uses recursive methods for parsing regular expressions. Technically we can try to negotiate it by using possessive quantifiers in regular expression.

The problem is in the first part of aciRegex. Here is synthetic test that reproduces the problem with original regexp and normal execution with adjusted regexp:
String originalRegex = "^\s*(\(\s*(\w+)\s*(!?=)\s*"([^\"]+)"\s*\)\s*)\s";
Pattern.matches(originalRegex, inputAci); <-- This fails with StackOverflowError
String modifiedRegex = "^\s*+(\(\s*+(\w++)\s*+(!?=)\s*+"([^\"]+)"\s*+\)\s*+)+\s+";
boolean res = Pattern.matches(modifiedRegex, inputAci); <-- executes normally
System.out.println("res = " + res);

Can you please validate it and provide resolution?

To Reproduce
Steps to reproduce the behavior:

1. Create ACI string as described above
2. Call Aci.decode(aciByteString, DN.rootDN())
3. Got StackOverflowError

Expected behavior
AciException is expected as ACI is not valid.

[vharseko]

Hi @myandrews, thanks for the detailed report and the suggested fix — the possessive-quantifier approach was exactly right.

This is now addressed in PR #666:

• AciTargets.targetRegex/targetsRegex (the pattern from your reproducer) now use possessive quantifiers (\s*+, \w++, (...)*+), so the regex engine matches the repetition iteratively instead of recursing once per repeated target — match results and capture-group numbering are unchanged.
• The same-shaped list regexes were hardened too: oidListRegex (Aci), attrListRegex (TargetAttr), rightsRegex (Permission).
• As defense in depth, Aci.decode now catches StackOverflowError and rethrows it as AciException (with the original error preserved as the cause), so a pathological ACI is treated as an invalid value instead of killing the worker thread.
• A regression test (AciDecodeStackOverflowTestCase) runs the decode on a bounded 256 KB stack thread: 10 000 repeated (targetscope="&O") targets are now rejected with AciException, and valid multi-target ACIs still decode.

Could you please verify the fix against your fuzzing setup — ideally on your original failing input? A build from the vharseko:issues/665 branch (or the PR artifacts once CI completes) should now throw AciException for that ACI. Any feedback on the PR is welcome.

[myandrews]

@vharseko thanks for extremely quick response. I'll check it on our fuzzing tests, probably tomorrow and return with results.

[myandrews]

@vharseko fuzzing tests regarding repetitive targets are passed. Fix works. There is another small problem regarding parsing userdn, I'll create another ticket.