clang import: reject AST nodes without the ext tokens the handlers read

[Nussu06]

Follow-up to #8939. Several createTokens handlers call mExtTokens.back()/front() or index relative to size() without checking that the node line had any ext tokens, so a --clang dump line with nothing after the node name (e.g. `-IntegerLiteral followed by a space) reads out of bounds. The same happens for StringLiteral, CharacterLiteral, CXXBoolLiteralExpr, LabelStmt, EnumDecl, DeclRefExpr and others, for a VarDecl that only has an address (mExtTokens[typeIndex - 1] with typeIndex 0), and for a FunctionDecl ending in prev (dereferences end()).

The AstNode constructor now throws an AST internal error when the node has no ext tokens, and createTokensVarDecl / createTokensFunctionDecl throw the same error for the other two shapes. Real clang dumps always carry the node address, location and type, so they are unaffected (checked --clang on the samples and a few lib files). crash2 now expects the internal error instead of silently producing no tokens. The size()-2 sites from #8939 are not touched.

I don't have Trac access, could someone create a ticket for this so I can reference it in the title?

[dmcppcheck]

There should probably be a trac ticket for this PR. Please create it, or ask us to create it. Then the PR title should be updated.

[dmcppcheck]

AI review

The PR does not fully fix the address-only case. It rejects a VarDecl whose line holds only an address, but a FunctionDecl (also CXXMethodDecl / CXXConstructorDecl) with the same shape still segfaults. getSpelling() returns "" because typeIndex <= 0, then addtoken(tokenList, "") adds no token and returns tokenList.back(). That is nullptr when the list is empty, and nameToken->scope() dereferences it at lib/clangimport.cpp:1411. When there are earlier tokens, addtoken instead returns the previous token, which is wrong but does not crash. EnumConstantDecl 0x1 reaches the same path at :1058.

The size()-2 sites (GotoStmt, NamespaceDecl, createTokensForCXXRecord) also still read out of bounds when a node has exactly one ext token. The description says these were left alone on purpose, but the title promises more than the change does. I found nothing wrong in the checks that were added, and they don't affect valid dumps.

Testing

• I built cppcheck and testrunner in pr/. ./testrunner TestClangImport reported 99 tests, 0 failed (1 todo), and TestGarbage TestCppcheck also passed. No clang is installed here, so test/cli/clang-import_test.py and real --clang runs could not be tested.
• Fuzzing: I made a copy of pr/ built with -D_GLIBCXX_ASSERTIONS and added a temporary fuzz test. It feeds every node type in clangimport.cpp lines with short ext-token lists (0x1, x, 'int', 0x1 <col:1>, 0x1 prev, …), both at top level and inside a function body.
• Reproducer test: I also added a fromFile test that calls parse() on a dump file. A valid dump prints int f ( ) ;, and `-VarDecl 0x1 now gives InternalError: invalid AST node: VarDecl, as intended.

Reproduced crash (address-only FunctionDecl):

$ printf '`-FunctionDecl 0x1\n' > d2.txt
$ DUMP=d2.txt ./testrunner TestClangImport::fromFile   # fromFile = parse(<contents of $DUMP>)
Segmentation fault (core dumped)
#1 clangimport::AstNode::addtoken (str="", valueType=true) at lib/clangimport.cpp:616
#2 clangimport::AstNode::createTokensFunctionDecl at lib/clangimport.cpp:1410

`-CXXConstructorDecl 0x1 and `-CXXMethodDecl 0x1 crash the same way. So does FunctionDecl 0x1 followed by a CompoundStmt child.

Still crashing on the size()-2 sites (the description says these were left alone):

`-FunctionDecl 0x1 <a.cpp:1:1, col:20> col:5 f 'int ()'
  `-CompoundStmt 0x2 <col:9, col:20>
    `-GotoStmt 0x3

This gives a segfault. ClassTemplateSpecializationDecl 0x1 segfaults at lib/clangimport.cpp:1519 (mExtTokens[mExtTokens.size() - 2]).

The fuzzer also hit many out-of-bounds reads on children (for example ConditionalOperator or CXXForRangeStmt with too few children). Those are outside the scope of this PR and I did not chase them further.

Comments

lib/clangimport.cpp:1410

This still crashes for an address-only function node, which is the same shape this PR rejects for VarDecl. For `-FunctionDecl 0x1 (and CXXMethodDecl / CXXConstructorDecl), getSpelling() returns "" because typeIndex <= 0. addtoken(tokenList, "") then adds no token and returns tokenList.back(), which is nullptr here, and the next line dereferences it. Reproduced:

`-FunctionDecl 0x1

parse() → SIGSEGV in addtoken (clangimport.cpp:616) called from here.

To fix it, throw the same AST InternalError when the spelling is empty or mExtTokens.size() < 2, like you did in createTokensVarDecl. EnumConstantDecl 0x1 reaches the same addtoken(tokenList, getSpelling()) path at line 1058.

---

Automated review of e267a95 by Claude. The pull request was built and tested locally.