Conversation
… time AVRO-4314 added Avro name-grammar validation for record/fixed/enum names, field names and aliases, and message names, but two parse-time paths were left unvalidated: - Enum symbols read from JSON: EnumSchema.NewInstance only checked for duplicates and never called ValidateSymbolName (that ran only on the programmatic Create() path). An out-of-spec symbol was carried through and emitted verbatim as an enum member identifier by the code generator. - Protocol names: Protocol.Parse never validated the protocol name, which is emitted as generated class names. Validate enum symbols on the JSON parse path (surfacing a SchemaParseException with the JSON path, consistent with the duplicate-symbol error) and validate the protocol name via SchemaName.ValidateName (surfacing a ProtocolParseException, consistent with message-name handling). Adds tests for out-of-spec enum symbols and protocol names.
zcsizmadia
left a comment
There was a problem hiding this comment.
Thanks @iemejia, closing these two gaps makes sense. I tried a couple of inputs on this branch, and they're now rejected even though Java accepts them and main parses them today:
1. Dotted protocol names
Protocol.Parse(@"{ ""protocol"": ""org.foo.Bar"", ""types"": [], ""messages"": {} }");
// ProtocolParseException: Invalid protocol name: org.foo.BarJava's Protocol.setName splits on the last . into namespace and name, so this form is valid there. The repo has it in lang/java/{compiler,idl}/src/test/idl/input/bar.avpr. Could we validate only the part after the last dot? That matches what SchemaName does, which validates the simple name and intentionally skips the namespace. It might also be worth doing the check in the Protocol constructor, the way Message does it, so the programmatic path is covered too.
2. Non-ASCII enum symbols
Schema.Parse("{\"type\": \"enum\", \"name\": \"Test\", \"symbols\" : [\"Ärger\"]}");
// SchemaParseException: Invalid symbol name: ÄrgerValidateSymbolName is still the ASCII-only ^[A-Za-z_][A-Za-z0-9_]*$ regex. AVRO-4314 made SchemaName.ValidateName Unicode-aware to match Java's default UTF_VALIDATOR, which is also what Java applies to enum symbols. Could ValidateSymbolName delegate to SchemaName.ValidateName(symbol, "enum symbol")? That keeps the two rules in sync and also fixes the Create() path.
A side benefit: ValidateName quotes and escapes the offending value via Quote(). Both new error messages here embed the raw value (Invalid protocol name: {name}, and Invalid symbol name: {symbol} from the regex path). Given the injection-style test case, reusing the inner exception's message would keep control characters out of the text.
Could you add positive test cases for both ("org.foo.Bar" parses with the name intact, and a non-ASCII symbol parses) alongside the negative ones?
Last thing: out-of-spec symbols like "A-B" in existing schemas or data file headers that C# readers accept today will fail to parse after this. That's consistent with Java, but it's probably worth a line in the release notes.
What
AVRO-4314 added Avro name-grammar validation for record/fixed/enum names, field names and aliases, and message names. This closes two parse-time paths it left unvalidated, which are then emitted verbatim as identifiers by the C# code generator:
EnumSchema.NewInstanceonly checked for duplicates and never calledValidateSymbolName(that ran only on the programmaticCreate()path). An out-of-spec symbol (leading digit, space, punctuation, …) was carried through and spliced as an enum member identifier by the generator.Protocol.Parsenever validated the protocol name, which is emitted as generated class names.Fix
EnumSchema.NewInstance: callValidateSymbolNameon each symbol, surfacing aSchemaParseExceptionwith the JSON path (consistent with the neighbouring duplicate-symbol error).Protocol.Parse: validate the protocol name viaSchemaName.ValidateName(name, "protocol"), surfacing aProtocolParseException(consistent with how message-name validation is surfaced inMessage.Parse).Tests
Adds
TestBasiccases for out-of-spec enum symbols (leading digit / space / hyphen) and aTestInvalidProtocolNamecase set (leading digit / space / injection-style name). Full C# suite passes (1533 tests, 0 failures).JIRA: https://issues.apache.org/jira/browse/AVRO-4346