Abdallah, It's good to see that someone had a good look at APC-1711 and 1712. Below you find some responses to your observations. I'm x-posting to SG16 list, as the contributions will be discussed in the Red Bank meeting. John Abdallah Rayhan wrote:
I took a pass through APC-1711 and APC-1712 and I would like to post my analysis on the mailing list. I don't claim my analysis to be complete, however I tried to point out the main problems I came across while reading APC-1711 ABNF. My primary goal is the final ABNF and not the ASN.1 for two reasons. First, I am not an ASN.1 expert, so I would leave critiquing it to the ASN.1 experts. Second, it would be a big distraction to the IETF having the MEGACO ABNF at an advanced
stage with a set of testbenches (ANNEX G). Hence, I will try to focus on APC-1711 ABNF which I call ABNF1711 and MEGACO/H.248 ABNF which I call ABNF248.
To start with I have noted that the ABNF1711 does not take the ABNF248 as is but modifies it to suite ASN.1 encoding ( is ASN.1 so restrictive that it does not allow ABNF248 syntax ? )
In developing an ASN.1 to ABNF translation mechanism, you have to take into account that the ABNF that is generated must be unambiguous. One of the dangers of omitting tags (field names) is introducing ambiguity, for instance because a string value may happen to be the same as a tag. If you do an ABNF spec by hand, you can verify that this does not lead to ambiguity _for that ABNF specification_ but in a mechanical translation process this is much harder. That's why the mechanism proposed in APC-1712 puts restrictions on the use of "AS ?"
These discrepancies are found in: (I started noting the different PRs but then found myself marking almost every one.)
Well, one grammar can be specified by different sets of PRs. So different sets of PRs I don't find worrying. Differences in the grammar is what's important.
- Tagging every value. ABNF1711: PkgdName = LBRKT PkgItemID [ PkgName ] RBRKT PkgName = LBRKT "/" ( ( "PE" EQUAL Name ) / "*" ) RBRKT PkgItemID = ( ( "PI" EQUAL Name ) / "*" )
ABNF248: pkgdName = [ (PackageName | "*") SLASH ] (ItemID | "*" ) PackageName = NAME ItemID = NAME
You need "PE=" before package name and "PI=" before ItemID. These differences appear in many production rules.
Yes, this is true. But I don't think it is a major change because the structure isn't changed very much.
- Extra brackets as illustrated in the previous example. In ABNF1711, there is a curly bracket before and after PkgdName while ABNF248 does not have that. This is all over ABNF1711.
There are indeed many instances where the generated ABNF has more braces. This is because braces are used to delimit sequences of parameters, consistently throughout the entire syntax. A packaged name is a sequence of two parameters: package name and item name. Hence it is surrounded by braces.
- Extensions do not appear any where in ABNF1711 while they are a major feature of ABNF248.
The ASN.1 has extension markers (the ellipses). These are not translated into ABNF, because the indicate that optional parameters may be added. If it is important, every extension marker in the ASN.1 of APC-1711 can be preceded by a sequence to produce the extension fields in the ABNF.
-ABNF1711 changes the syntax of the MEGACO constructs.
ABNF1711: RequestedActions = LBRKT RequestedAction [ "E" EQUAL SecondEventsDescriptor ] [ "SG" EQUAL SignalsDescriptor ] [ "IC" ] RBRKT
ABNF248: requestedActions = requestedAction LWSP [COLON LWSP embeddedSignalEvents ] [COLON LWSP InterceptToken ] embeddedSignalEvents = firstEmbedding [COMMA firstEmbedding]
as you can see the first level of embedding (firstEmbedding) is chopped in ABNF1711. As a matter of fact, it changes the syntax into something different.
There's no more production for firstEmbedding, but in ABNF248, the syntax continues: firstEmbedding = (secondEvent | signalsDescriptor) secondEvent = EventsDescriptorToken EQUAL [...] signalsDescriptor = SignalsDescriptorToken [...] Noting that the EventsDescriptorToken is an "E" and that the SignalsDescriptorToken is the string "SG", you see that the differences are very minor. ABNF1711 does not use colons for separators, but I think that's not important.
-ABNF1711 enforces ordering of descriptors and parameters while the ABNF248 does not. This issue was raised before and the resolution was made to enforce no order since there is no relationship between descriptors, e.g., modems or muxes, whichever comes first does not affect the other.
True. This is not a problem of the approach per se, but my personal stubborness. ABNF1711 enforces ordering, but the syntax specifies that every descriptor can only occurs once in a list, while ABNF248 puts that in a comment (i.e., specifies it as a semantic constraint). It's a small operation to write ASN.1 that will produce ABNF that contains "descriptor *(COMMA descriptor)" constructs instead of the "[descr1][descr2][descr3]" constructs.
- ABNF248 uses COMMA as a separator for commands, descriptors and parameters. This concept is gone in ABNF1711.
If we rewrite as explained above, you'll have your commas back.
- ABNF1711 changes the syntax and the semantics of MEGACO, e.g., ABNF1711: NotifyReply = LBRKT TerminationID [ "ER" EQUAL ErrorDescriptor ] RBRKT You always have TerminationID regardless of whether it is NotifyReply error or not, and ErrorDescriptor is tagged.
ABNF248 notifyReply = NotifyToken ( commandError | [EQUAL TerminationID]) commandError = EQUAL TerminationID LBRKT errorDescriptor RBRKT TerminationID is optional in pure notify reply and mandatory in notify error. ErrorDescriptor is not tagged.
I explained about the tagging above, so let that go. The optional TerminationID presented a bit of a problem for the ASN.1. Because I couldn't for the life of me remember why it's important to be able to leave TerminationIDs out in NotifyReply, I made them mandatory.
Also, ABNF1711: SecurityParmIndex = ( 1( BASE64 ) BASE64END ) SequenceNum = ( 1( BASE64 ) BASE64END ) AuthData = ( 5( BASE64 ) BASE64END ) BASE64 = 4( BASE64CHAR ) BASE64END = 4( BASE64CHAR ) / ( 3( BASE64CHAR ) "?" ) / ( 2( BASE64CHAR ) "??" ) Where did "?" and "??" come from ?
They ensure that the number of characters in the BASE64 encoded octet string is a multiple of 4. I was told that BASE64 works that way.
ABNF248: SecurityParmIndex = 8(HEXDIG) SequenceNum = 8(HEXDIG) AuthData = 32*64(HEXDIG)
These are some of the differences. If we review each PR, no PR in ABNF1711 will be saved. So I will stop here.
Once again, the PRs aren't the most important, the differences in grammar are.
I dont know if the kind of constraints ABNF1711 enforces on the ABNF are due to to the ASN.1 encoding constraints or because of ASN.12ABNF. Whatever the reasoning is, APC1711 should focus on completing the ASN.1 description and generate similar testbenches as in the MEGACO I-D either binary or text, or both. The binary will be definitely different than the text because of tagging, however the text should be same as the test benches written so far. In theory, bin and text should reflect the same protocol and should introduce the least entropy into the syntax.
My recommendation is that rather than writing ASN.12ABNF tool, APC1711
should write examples that use the ASN.1 and bin2text and text2bin tools in order to test the ASN.1 encoding. Once that is out of the way and we know that the ASN.1 is ok, we could look into tools like ASN.12ABNF presuming that it would generate ABNF that is equivalent to ABNF248 syntactical and semantical wise. Otherwise, ABNF1711 approach is unproductive and waste of effort trying to verify every PR of ABNF1711.
I agree that examples are useful, but this comment is counter productive.
In addition to that, if we assume that the ASN.1 is ok and the ASN.12ABNF is ok, from what I saw both are not,
Well, if you define ABNF248 as being right and anything that's different as being wrong, then obviously your right, right? Otherwise, your're wrong, right? As far as I can tell, the ASN.1 is ok, but it definitely needs review. I'm glad you took the effort to be the first to comment. And I'm glad that you didn't find any semantic errors (barring the mandatory TerminationID, which I don't consider an error but a difference).
is there any guarantee that when they change the ASN.1, that will not introduce problems into the ABNF ? My feeling is that it would be a source of confusion and we would need to review the ABNF and probably hand craft some PRs to make it consistent and ambiguous free.
If that is that is the case, why should we throw out ABNF248 which we have tested and have a high degree of confidence in. We are very familiar with ABNF248; we know the ins and outs of it; and we know how to fix any problem without disturbing the rest of the grammar. How would ABNF1711 do that ?
Using the mechanical translation mechanism gives me a higher degree of confidence. I don't have to worry anymore about having accidentally omitted a closing brace, or being inconsistent with the use of separators.
As for ASN.1, it is not compatible with the standard. This means that ASN.1 compilers can not produce a correct code.
APC-1712 addresses this. The extensions that were added to deal with the different types of separators and brackets in ABNF248 can be put into special comments in the ASN.1. They take the form "--& <extension> --" A tool that does the text encoding as described, recognizes these comments, a normal ASN.1 compiler sees them as comments, ignores them and produces code (correctness is up to the compiler writer :-).
regards Abdallah Rayhan
-- John Segers email: jsegers@lucent.com Lucent Technologies Room HE 306 Dept. Forward Looking Work phone: +31 35 687 4724 P.O. Box 18, 1270 AA Huizen fax: +31 35 687 5954