CESNET

lhotka@cesnet.cz

Plantronics

rohan@ekabal.com

Nortel

schishol@nortel.com

Operations and Management NETMOD This draft describes the algorithm and rules for defining NETCONF data modelss using Document Schema Definition Languages (DSDL) with additional annotations and based on a mapping from YANG.

The NETCONF Working Group has completed a base protocol used for configuration management. This base specification defines protocol bindings and an XML container syntax for configuration and management operations, but does not include a modeling language or accompanying rules for how to model configuration and status information (in XML syntax) carried by NETCONF. The IETF Operations area has a long tradition of defining data for SNMP Management Information Bases (MIBs) using the SMI to model its data. While this specific modeling approach has a number of well-understood problems, most of the data modeling features provided by SMI are still considered extremely important. Simply modeling the valid syntax rather than additional semantic relationships has caused significant interoperability problems in the past. The NETCONF community concluded that a data modeling framework is needed to support ongoing development of IETF and vendor-defined management information modules. The NETMOD Working Group was chartered to address this problem, by defining a new human-friendly modeling language based on SMIng and called YANG. Since NETCONF content contain XML data, it is natural to express the constraints on NETCONF content using standard XML schema languages. For this purpose, the NETMOD WG selected the Document Schema Definition Languages (DSDL) that is being standardized as ISO/IEC 19757 . The DSDL framework comprises a set of XML schema languages that address grammar rules, semantic constraints and other data modeling aspect but also, and more importantly, do it in a coordinated and consistent way. While it is true that some DSDL parts have not been standardized yet and are still work in progress, the two crucial parts that the YANG-to-DSDL mapping relies upon - RELAX NG and Schematron - already have the status of an ISO/IEC International Standard and are supported in a number of software tools. This document contains the specification and a mapping that translates YANG data models to an XML schema (or multiple schemas) utilizing a subset of the DSDL schema languages together with a limited number of other annotations. The resulting schema(s) can be used (possibly after further processing) for validating NETCONF content and making sure that they are compliant to the original data model. Depending on the architecture of a particular NETCONF implementation, the DSDL schemas may be used directly for validating NETCONF content using existing XML tools and libraries. But even if the schemas are not used for practical validation, the YANG-to-DSDL mapping described in this document should be regarded and perused by implementers as a formal specification of the correspondence between a YANG data model and NETCONF content. In the text, we use the following typographic conventions: YANG statement keywords are delimited by single quotes. Literal values are delimited by double quotes. XML element names are delimited by "<" and ">" characters. Names of XML attributes are prefixed by the "@" character.

The mapping described in this document uses three of the DSDL schema languages, namely RELAX NG, Schematron and DSRL. In addition, three types of annotations are used in the RELAX NG schema. Two of them are standard annotation vocabularies: Dublin Core metadata terms serve for recording metadata and RELAX NG DTD compatibility annotations are used for documentation strings. Finally, a small set of special annotations were designed in order to convey other semantic information that is important in the context of NETCONF content validation. The DSDL schema languages and annotations are described in the following subsections. Schema-independent Library – Parts of this solution that are the same for all Schema definitions. Conceptually these are written once and used many times. This includes NETCONF specific annotations to Relax NG (DML) and common Schematron rules. Schema-specific Definitions – Parts of this solution that are unique to a specific Schema definition. These are generated or written for each Schema definition

----------------------- | Schema-independent | | Library | | |---------| ------------------------ | |--> Validate ------------------- | NETCONF | Schema-specific | | Content ---> | Definitions |-------------- | | -------------------

RELAX NG is an XML schema language for grammar-based validation. Like the W3C XML Schema language , it is able to describe constraints on the structure and contents of XML documents. Unlike the DTD and XSD schema languages, RELAX NG intentionally avoids any infoset augmentation such as defining default values. In the DSDL architecute, this task is delegated to other schema languages, in particular DSRL . As its base datatype library, RELAX NG uses the W3C XML Schema Datatype Library , but unlike XSD, other datatype libraries may be used along with it or even replace it if necessary. RELAX NG is very liberal in accepting annotations from other namespaces. With few exceptions, such annotations may be placed anywhere in the schema and need no encapsulating element such as <annotation> in XSD. RELAX NG schema can be represented using two equivalent syntaxes: XML and compact. The compact syntax is described in Annex C of the RELAX NG specification , which was added in 2006 (Amendment 1). Automatic bidirectional conversions between the two syntaxes can be accomplished using for example Trang. For its terseness and readability, the compact syntax is often the preferred form for publishing RELAX NG schemas whereas validator and other software tools generally require the XML syntax. The weak point of the compact syntax is the handling of annotations. While in the XML syntax the annotating elements and attributes are represented in an uniform way, the compact syntax uses four different syntactic constructs (documentation, grammar, initial and following annotations). As a result, the impact on readability resulting from adding annotations is often much stronger for the compact syntax than for the XML syntax. Since the mapping from YANG to DSDL relies heavily on annotations, this document uses exclusively the RELAX NG XML syntax, which is more appropriate for further machine processing. Where appropriate, though, the schemas resulting from the translation MAY be presented in the equivalent compact syntax.

While the traditional schema languages such as DTD, XSD or RELAX NG are based on the concept of a formal grammar, Schematron utilizes a rules-based approach. Its rules may specify arbitrary conditions involving data from different parts of an XML document. Each rule consists of three essential parts: Context - an XPath expression that defines the set of locations where the rule is to be applied, Assert or report condition - another XPath expression that is evaluated relative to the location matched by the context expression. Human-readable message that is displayed when the assert condition is false or report condition is true. The difference between the assert and report condition is that the former is positive in that it states a condition that a valid document has to satisfy, whereas the latter specifies an error condition. The mapping described in this document uses exclusively the positive (assert) form. Schematron draws most of its expressive power from XPath and XSLT . ISO Schematron allows for dynamic query language binding so that the following XML query languages can be used: STX, XSLT 1.0, XSLT 1.1, EXSLT, XSLT 2.0, XPath 1.0, XPath 2.0 and XQuery 1.0 (this list may be extended in future). The human-readable error messages are another feature that distinguishes Schematron from other schema langauges such as RELAX NG or XSD. The messages may even contain XPath expressions that are evaluated in the actual context and thus refer to XML document nodes and their content. ISO Schematron introduced the concept of abstract patterns whose purpose is similar to functions in programming languages. The mapping described in this document uses abstract patterns for defining common constraints such as uniqueness of certain leaf values in list items. The rules defined by a Schematron schema may be divided divided into several subsets known as phases. Validations may then be set up to include only selected phases. In the context of NETCONF data validation, this is useful for relaxing constraints that may not always apply. For example, the reference integrity may not be enforced for a candidate configuration.

Schematron can either be defined in standalone specifications or embedded in the RELAX NG Schema. This solution is just using standalone Schematron. Design Alternative: Use standalone Schematron to create a library of common rules used to validate any content. and use embedded rules to define Schema specific rules.

This section outlines the Schematron rules used to validate all NETCONF content.

The Schematron validator evaluates the context of each rule, then evaluates each assert or report clause within the rule. The following is used for insuring that a key is unique within a list context.

The value ifIndex must be unique among all interfaces. ]]>

For every ifIndex element, the 'assert' tests if the number of interfaces which have an ifIndex matching the current one (being evaluated) is one. This XPath expression is rather long. Since we will use this idiom repeatedly, we can define a Schematron abstract pattern to express this more succinctly:

The key "" needs to be unique within the list at: . ]]>

Likewise, we can write Schematron abstract patterns for validating keyref relationships as well.

The contents of "" must be a '' with the key "" in this context: . ]]>

Even the Schematron abstract patterns shown above contain paths that a computer could derive automatically. The XSLT stylesheet described in [Editor's Note: not yet in document] could read the data model processing annotations to automatically generate Schematron patterns that use these abstract patterns. The XSLT stylesheet is just calculating the context XPaths by examining the document structure in the RelaxNG schema. Once generated, these rules can be used to test the validity of instance documents. These Schematron tests can provide different levels of validity checking.

Conceptually, an implementation can validate its instance data in different logical phases, adding more tests with each phase. We use three levels or phases for validating an instance document. There is a level of validation which is appropriate even for loose XML document fragments which still maintain their hierarchy (the fragment phase), there is a level of validation appropriate for a cohesive XML document but which may not be able to validate relational integrity checks against some operational state (the standard phase), and there is validation which includes all relational integrity checks (the full validation phase). For example, in Netconf an edit-config operation can cause the replacement a small fragment of XML. A candidate configuration may be waiting for application but can't check the readiness of a piece of hardware that the configuration refers to. From the NETCONF perspective, these three phrases can considered to have the following scope and triggers: the fragment phase: This can be run against individual NETCONF operations; should be automatically triggered the standard phase: This can be run against the candidate configuration, but won’t always pass; should be manually triggered the full validation phase: This can be run against a running configuration; should be automatically triggered [Editor's Note: open discussion point on whether all stages, including fragmentation, apply or just the last two via a conceptual tree model. Sharon feels that should be validatable in the tool-friendly language.] During the Fragment phase validation: Verify that the content is appropriate to the operation (by passing a variable to Schematron with the type of operation). During Standard phase validation (all rules except for keyref checking): Verify that mustUse items are present Check the uniqueness for unique and key annotations Print a warning if any manual validation rules are present During Full phase validation: add keyref checks. All embedded Schematron will run in the standard phase unless the a 'dml:phase' attribute is included with the name of a different phase ('fragment' or 'full').

]]>

DSRL (pronounced "disrule") is Part 8 of DSDL in the stage of FCD (Final Committee Draft). Unlike RELAX NG and Schematron, it specifically designed to modify XML information set of the validated document. The primary application for DSRL is renaming XML elements and attributes. DSRL can also define default values for XML attributes and elements so that elements or attributes with these default values are inserted if they are missing in the validated documents . This latter feature is used by the YANG-to-DSDL mapping for representing YANG defaults for leaf nodes.

Besides RELAX NG, Schematron and DSRL, the YANG-to-DSDL mapping utilizes three sets of annotations that carry metadata, documentation strings and additional semantics of YANG data models that cannot be fully expressed using the above schema languages. These annotations may be attached to the elements in the RELAX NG schema as either subelements or attributes from different namespaces (see for details).

Dublin Core is a system of metadata terms that was originally created for describing metadata of World Wide Web resources in order to facilitate their automated lookup. Later it was accepted as a standard for describing metadata of arbitrary resources This specification uses the definition found in .

See http://relaxng.org/compatibility-20011203.html These annotations are part of the RELAX NG DTD Compatibility specification . The YANG-to-DSDL mapping uses only the <documentation> annotation for representing YANG 'description' and 'reference' strings, except at the top level of a module where Dublin Core metadata terms are used for this purpose. Note that there is no intention to make the resulting schemas DTD-compatible, the main reason for using this annotation is technical: it is supported and adequately interpreted by many RELAX NG tools.

This specification introduces two XML attributes that are convey special semantics that cannot be fully expressed in the standard DSDL schema languages.

When the cardinality of an element is more than one (ie, it is in a list), list keys are important for defining the behavior of the NETCONF <edit-config> method (see , Sec. 7.8.6): Every operation addressina list item must provide all key nodes and the contents of the keys cannot be changed. A detailed explanation is provided in . The uniqueness constraint can be expressed as a Schematron rule in the same way as for the 'unique' statement (see ).

The 'config' annotation indicates the category of data that applies to the annotated element and all its children until another 'config' annotation. The content of the 'config' element is an boolean, where true is the default value. It indicates that the covered data is configuration and that it is at least theoretically possible to include this data in read, write, create, and delete operations. Likewise non-configuration information can be read, but cannot be included in write, create, and delete operations. See for details.

[Editor's Note: Note this is in support of client-side conformance which may not yet be in yang. It is different from the built in <optional> and <zeroOrMore> type tags in RelaxNg, which are used for server-side conformance in this solution] An element in the data model tagged with the 'mustUse' annotation indicates that the tagged element needs to appear in the instance document whenever its parent appears. Items with a default value annotation cannot also have a mustUse annotation. In many schemas, there is a facility for using fragments or patches of XML documents. (Netconf uses these fragments extensively in edit-config operations for example). In order to accommodate these fragments, the cardinality of an otherwise "required" element may allow the element to be optional in an XML fragment. The mustUse annotation provides a way to express what is actually required in this situation. See also the client-side conformance discussion in on conformance.

When defining NETCONF content, it is also necessary to define machine-readable conformance for that content. The conformance method described provides a means of providing information about individual elements which is then used to calculate the schema conformance. There is no separate definition of schema conformance. Previous solutions with separate conformance sections were found to have issues, particularly in keeping them up to date as the schema evolved. They also were not always used outside of standards activities where people did not either fully understand them or see the value in them. Conformance specifies not only whether to object must be supported, but also the level of access, read versus write for example that is minimally required.

When defining attributes and elements in the XML syntax, the 'optional', 'oneOrMore' or 'zeroOrMore' tags are used to specify the cardinality of the element. In the compact syntax, "?" means optional, "+" means one or more, "*" means zero or more. When no cardinality indicator is present, this is interpreted to mean exactly once.

Operations that can be performed on managed objects fall into one of the following equivalence classes: "create", "delete", "read", "write", and "execute". A value of "create" means it is possible to create new instances of this element. A value of "delete" means it is possible to destroy instances of this element. A value of "read" means that it is possible to view values of this element. A value of "write" means it is possible to modify an instance of this element. A value of "execute" means that there is a side effect execution such as rebooting that is permissible as a result of the command. The kind of access which is allowed for a particular element depends on its dml:config value. The minimum access needed for an implementation for a specific kind of dml:config is that element's 'minAccess'. The maximum access allowed is that element's 'maxAccess'. Normally these values do not need to be modified. There is a default access class for each dml:config. Data Category minAccess maxAccess config read read,write,create,delete non-config read read However, if a specific piece of data needs a different class it can be modified using the config annotation's minAccess and maxAccess attributes. For example, perhaps a 'controller' element can only be created automatically as a side-effect of some other action. The example below documents this by explicitly setting the maxAccess attribute: [Editor's note: this minOccurs/maxOccurs approach is not exactly what is used in yang. A common approach needs to be agreed.]

a bunch of sub-elements of controller ... true ]]>

To describe how this applies specifically to NETCONF: a value of "create" means it is possible to create new instances of this element using commands like the NETCONF 'edit-config' or copy- config' commands. A value of "delete" means it is possible to destroy instances of this element using commands like the NETCONF 'edit-config', 'copy-config' or 'delete-config' operations. A value of "read" means that it is possible to view values of this element using commands like the 'get-config', 'get' or 'notification' operations. A value of "write" means it is possible to modify an instance of this element using commands like the NETCONF 'edit-config' or 'copy- config' commands. A value of "execute" means that there is a side effect execution such as rebooting that is permissible as a result of the command. For example, commands like the NETCONF 'edit-config' or a 'copy-config' command or the ability to execute a commands like the 'lock', 'unlock' or 'kill-session' command.

As a schema evolves, certain elements and attributes may no longer be relevant. Simply deleting these from the schema may be acceptable for elements that did not see implementation, but others will require a strategy to allow implementers to migrate away from the old elements. An optional processing annotation called "status" SHOULD be used to provide the status of the element. When not present, it will assume a value of "current". The other value of this object are "deprecated" and "obsolete". A value of "deprecated" indicates that implementations should consider migrating away from this object and that its implementation is no longer required to be considered conformant. A value of "obsolete" means the object should not be implemented. Deprecated and obsolete content is never removed from the document and its element name can never be re-used.

<dml:status>current</dml:status>

Additional information about conformance should be specified using a documentation tag. Examples of additional conformance information that may be useful to provide includes how implementations can specify specific exceptions to required conformance, dependencies between elements (in order to do A, you need to first do B) and conditional conformance (if BGP, then ...). [Editor's Note: This may need rationalization with yang]

In order to claim compliance to a schema, all elements and attributes need to conform to their given cardinality definitions and all elements and attributes with a status of "current" and with a cardinality greater than or equal to one need to be supported. In addition, all of the operations listed by the minAccess attribute need to be supported.

Client-side conformance is a method of indicating whether presence of an object is required in order to be a valid configuration. A new processing annotation is added called mustUse to support this. When present, this object is required in a valid configuration and when not present, it is optional in a valid configuration. Note that optional objects may have default values to enable them to have values in the configuration without being explicitly set by the client.

<dml:mustUse/>

This section defines the Relax NG Schema for the NETCONF specific annotations used in this solution. [Editor's Note: some parts of this may need to be rationalized with section 4 of this document.]

1.0 config status statistics action notify config read write create delete execute false any-order user-order any-order [^: \n\r\t]+ ([a-zA-Z][a-zA-Z0-9\-\+\.]*:|\.\./|\./|#).* active deprecated obsolete active fragment std full false ]]>

YANG modules can be mapped to DSDL schemas in a number of ways. The mapping prosedure described in this document uses several specific design decision that are discussed in the following subsections.

DSDL schemas generated from a YANG module using the procedure described in this document are intended to be used for validating contents of NETCONF protocol data units that are exchanged between the client and server. However, every YANG model generates an array of possible PDUs that cannot be described by a single set of DSDL schemas: First, besides configuration and status data, a YANG module may also define signatures of new NETCONF RPC methods and notifications. Various subtrees of configuration and/or status data may appear in different NETCONF protocol operations - <get>, <get-config> and <edit-config>, each of them having specific rules for their content. Finally, contents of replies to the <get> and <get-config> may be modified by filters (based either on subtree filtering specified in or XPath expressions ) On the other hand, variability of PDU contents resulting from and is limited to selecting one or more specific subtrees from the full data tree specified by a YANG module. In order to avoid technical complications with specifying the context for a particular PDU, we introduce the notion of conceptual data tree containing everything that is defined by one or more YANG module that are supposed to be used together in a NETCONF session: full configuration and status data tree, RPC messages and the corresponding replies, and notifications. For YANG module "foo", the conceptual data tree may be schematically represented as follows:

... configuration and status data ... ... ... ... ... ... ... ... ... ]]>

The namespace URI "urn:ietf:params:xml:ns:netmod:tree:1" identifies a simple vocabulary consisting of a few elements that encapsulate and separate the various parts of the conceptual data tree. The purpose of this construction is to establish a single well-defined target for the mapping from YANG to DSDL - its output is a DSDL schema specifying formally the constraints that the conceptual data tree has to satisfy in order to be compliant with the data model that served as a preimage for the DSDL schema. The drawback of this approach is that the DSDL schema(s) generated by the mapping cannot be directly used for validating any real-life NETCONF PDU. Nonetheless, given a specific context, such as reply to <get-config> with a certain filter, it should be reasonably straightforward to derive the DSDL schema for validating this particular PDU from the conceptual data tree schema, for example by using XSLT. However, this subsequent transformation is outside the scope of the present document. Incidentally, by introducing the conceptual data tree we are also able to resolve the difficulties stemming from the fact that a single YANG module may define multiple parallel data hierarchies, which is something that XML does not allow. In the conceptual data tree, the multiple hierarchies can be easily accomodated under the <nmt:data> element.

Both YANG and RELAX NG offer means for modularity, i.e., for splitting the contents into separate modules (schemas) and combining or reusing them in various ways. However, the approaches taken by YANG and RELAX NG differ. Modularity in RELAX NG is intended more for ad hoc combinations of a small number of schemas whereas YANG assumes a large set of modules similar to SNMP MIBs. The following differences are important: IN YANG, when module A imports module B, it gets access to the definitions (groupings and typedefs) appearing at the top level of module B. However, no part of module B's data tree is imported along with it. In contrast, the <include> pattern in RELAX NG imports both definitions of named patterns and the entire schema tree from the included schema. The names of imported YANG groupings and typedefs are qualified with the namespace of the imported module. On the other hand, the data nodes contained in an imported grouping, when used in the importing module, become part of the importing namespace. In RELAX NG, the names of patterns are unqualified and so named patterns defined in the importing and imported module share the same flat namespace. The contents of RELAX NG named patterns may either keep the namespace of the schema where they are defined or inherit the namespace of the importing module, analogically to YANG. However, in order to achieve the latter behavior, the imported module must be prepared in a special way as a library module that cannot be used as a stand-alone schema. Hence, the conclusion is that the modularity mechanisms of YANG and RELAX NG, albeit similar, are incompatible. Therefore, the design decision for the mapping algorithm was to collect all definitions in a single DSDL module even if in YANG it is distributed in several modules. In other words, translations of imported groupings and typedefs are installed in the translation of importing module - but only if they are really used there. However, the 'include' statement that can be used in YANG to include submodules has the same semantics as the <include> pattern of RELAX NG: th esubmodule contributes both definitions and data nodes and inherits the namespace of the parent module. Consequently, the structure of YANG submodules belonging to the same module can be retained in RELAX NG.

RELAX NG supports different styles of structuring the schema: One extreme, often called "Russian Doll", specifies the structure of an XML instance document in a single hierarchy. The other extreme, the flat style, uses a similar approach as the Data Type Definition (DTD) schema language - every XML element is introduced inside a new named pattern. In practice, some compromise between the two extremes is usually chosen. YANG in principle supports both styles, too, but in most cases the modules are organized in a way that's closer to the "Russian Doll" style, which provides a better insight into the structure of the configuration data. Groupings are usually defined only for contents that are prepared for reuse in multiple places via the 'uses' statement. In contrast, RELAX NG schemas tend to be much flatter, because finer granularity is in RELAX NG also needed for extensibility of the schemas - it is only possible to replace or modify schema fragments that are factored out as named patterns. For YANG this is not an issue since its 'augment' statement can delve, by using path expressions, into arbitrary depths of existing structures. In general, it not feasible to map YANG extension mechanisms to those of RELAX NG. For this reason, the mapping essentially keeps the granularity of the original YANG data model: definitions of named patterns in the resulting RELAX NG schema usually have direct counterparts in YANG groupings and definitions of complex types.

Validation of the conceptual tree data may occur in different logical phases, adding more tests with each phase. We use three levels or phases for validating an instance document. There is a level of validation which is appropriate even for loose XML document fragments which still maintain their hierarchy (the fragment phase), there is a level of validation appropriate for a cohesive XML document but which may not be able to validate relational integrity checks against some operational state (the standard phase), and there is validation which includes all relational integrity checks (the full validation phase). For example, in NETCONF an edit-config operation can cause the replacement a small fragment of XML. A candidate configuration may be waiting for application but can't check the readiness of a piece of hardware that the configuration refers to. From the NETCONF perspective, these three phrases can considered to have the following scope and triggers: the fragment phase: This can be run against individual NETCONF operations; should be automatically triggered the standard phase: This can be run against the candidate configuration, but won’t always pass; should be manually triggered the full validation phase: This can be run against a running configuration; should be automatically triggered During the Fragment phase validation it is verified that the content is appropriate to the operation. During Standard phase validation (all rules except for keyref checking): Verify that mustUse items are present. Check the uniqueness for unique and key annotations. Print a warning if any manual validation rules are present. During Full phase validation: add keyref checks. All embedded Schematron will run in the standard phase unless the a 'dml:phase' attribute is included with the name of a different phase ('fragment' or 'full').

]]>

This section gives an overview of the YANG-to-DSDL mapping, its inputs and outputs, and presents the general goals of this specification. As described in , the mapping process is divided into two stages. This specification concentrates on the first stage, namely translating one or more YANG modules into a DSDL schema for the conceptual tree. The aim is to translate most of the semantics of the YANG module(s) and express them using the logic of DSDL schema languages. The schema for the conceptual tree captures grammatical and syntactic constraints and other information for the configuration datastore rendered as XML (which can be obtained inside the reply to NETCONF <get> method without any filters), and contents of both input and output parts of all RPC methods defined by the module(s). The second stage, which transforms the schema for the conceptual tree into multiple schemas for validating the contents of different NETCONF PDUs under specific circumstances is outside the scope of this specification and will be addressed in subsequent documents. An implementation of the mapping algorithm MUST accept as input one or more valid YANG modules that are already parsed into a tree data structure, henceforth denoted as the "YANG tree". Each node in the YANG tree is an object (structure) that corresponds to a single YANG statement. This object records the essential parameters of the statement (keyword, argument string) and keeps a pointer to the parent statement and a list of substatements. It is important to be able to process multiple YANG modules together since multiple modules may be negotiated for a NETCONF session and the contents of the configuration datastore is then obtained as the union of data trees specified by individual modules, perhaps with multiple root nodes. In addition, the input modules may be coupled by the 'augment' statement in which one module augments the data tree of another module. It is also assumed that the algorithm has access, perhaps on demand, to the parsed form of all YANG modules that the module imports (transitively). The output of the mapping algorithm is a RELAX NG schema annotated with XML elements and attributes of the other DSDL schema languages (Schematron and DSRL) and additional annotations (metadata, documentation and NETMOD-specific attributes). Therefore, the resulting schema is an XML document containing qualified names from the following vocabularies and namespaces: RELAX NG (DSDL Part 2 ) is used for representing grammatical constraints, simple datatypes (via the W3C XML Schema Datatype Library ), complex datatypes, restrictions of datatypes, and YANG groupings. RELAX NG namespace is "http://relaxng.org/ns/structure/1.0" and the namespace of the W3C Schema Datatype Library is "http://www.w3.org/2001/XMLSchema-datatypes". Schematron (DSDL Part 3 ) is used for specifying additional semantic constraints that are either inherent to YANG (e.g., uniqueness of list keys) or specified by the data model author (e.g., using the 'must' statement). Schematron namespace is "http://purl.oclc.org/dsdl/schematron". Document Schema Renaming Language (DSRL; DSDL Part 8 ) is used for specifying default values for YANG leaf nodes and typedefs (cf. 'default' statement). DSRL namespace is "http://purl.oclc.org/dsdl/dsrl". Dublin Core metadata terms are used for general data model metadata such as authorship information, revision date or model description. The namespace for Dublin Core metadata terms is "http://purl.org/dc/terms". RELAX NG DTD compatibility annotations are used for 'description' and 'reference' strings, except at the top level of a module where Dublin Core metadata terms are used for this purpose. The namespace for DTD compatibility annotations is "http://relaxng.org/ns/compatibility/annotations/1.0". NETMOD-specific annotations is a small collection of other annotations that convey additional semantics expressed by YANG model. They are always modeled as XML attributes attached to the RELAX NG elements that they qualify. The namespace for NETMOD-specific annotations is "urn:ietf:params:xml:ns:netmod:dsdl-attrib:1". An implementation SHOULD allow selection of a subset of schema languages and annotation types to be used for output. For example, a user might want to select pure RELAX NG without any annotations. The DSDL architecture assumes that each constituent schema language is confined to a separate schema document. In contrast, the schema for the conceptual tree produced by the mapping combines all DSDL languages and annotations into a single XML tree. This is considered more appropriate for the following reasons: The conceptual tree adds several artificial nodes. As a result, the XPath expressions that define the context for Schematron rules and DSRL default values differ from those that used in validation schemas and would thus have to be modified in each of the separate DSDL schemas. Putting all constraints in one place leads to a more compact representation that is also easier to read. The schema for the conceptual tree is not intended for validation. To illustrate the difference, consider the following YANG module:

module dhcp { namespace "http://example.com/ns/dhcp"; prefix dhcp; ... container dhcp { ... leaf max-lease-time { ... }; leaf default-lease-time { ... must '$this <= ../max-lease-time' { error-message "default-lease-time must be less than max-lease-time"; } default 600; } ... } ... }

If multiple schemas for the conceptual tree were generated, i.e., separate RELAX NG, Schematron and DSRL, the 'must' statement would be mapped to the following Schematron fragment:

default-lease-time must be less than max-lease-time ]]>

Similarly, the 'default' statement results in the following DSRL fragment:

/nmt:netmod-tree/nmt:data/dhcp:dhcp dhcp:default-lease-time 600 ]]>

In both the Schematron and DSRL fragments, the context for the rules is explicitly specified by means of an XPath expression, which however always contains the two top-level nodes of the conceptual tree. If we insert these rules as annotations into the RELAX NG schema, the context is implicitly defined by the location of the rules with the RELAX NG schema. For the validation schemas that are generated in the second stage of the mapping, the correct XPath expressions can be generated from this implicit information. Consequently, the resulting RELAX NG for the conceptual tree fragment is significantly simpler:

... The default-lease-time must be less than max-lease-time 600 ]]>

Since RELAX NG compact syntax is often the preferred form for perusal by human readers, the secondary goal of the mapping is to guarantee a reasonable level of readability of the resulting RELAX NG in the compact syntax as well, even if multiple embedded annotation types are used.

In YANG, all leaf nodes are optional unless they contain substatement

mandatory true;

or unless they are declared as list keys. Lists or leaf-lists are optional unless they contain 'min-elements' substatement with argument value greater than zero. A slightly more complicated situation arises for YANG containers. First, containers with the 'presence' substatement are therefore always optional since their presence or absence carries specific information. On the other hand, non-presence containers may be omitted if they are empty. For the purposes of the YANG-to-DSDL mapping, we declare a non-presence container as optional if and only if the following two conditions hold: none of its descendant leaf nodes is mandatory or, if such a leaf exists, it is enclosed in an intervening container with presence; none of its descendant list or leaf-list nodes has 'min-elements' substatement with argument value greater than zero or, if such a list or leaf-list exists, it is enclosed in an intervening container with presence. In RELAX NG, all elements that are optional must be explicitly wrapped in the <optional> element. The mapping algorithm thus uses the above rules to determine whether a YANG node is optional and if so, insert the <optional> element in the RELAX NG schema.

YANG groupings and typedefs are generally translated into RELAX NG named patterns. There are, however, several caveats that the mapping has to take into account.

YANG typedefs and groupings may appear in all levels of the module hierarchy and are subject to lexical scoping, see , Section 5.8. Moreover, top-level symbols from external modules are imported as qualified names consisting of the external module namespace and the name of the symbol. In contrast, named patterns in RELAX NG (both local and imported via the <include> pattern) share the same namespace and are always global - their definitions may only appear at the top level of the RELAX NG schema as children of the <grammar> element. Consequently, whenever YANG groupings and typedefs are translated into RELAX NG named pattern definitions, their names MUST be disambiguated in order to avoid naming conflicts. The mapping uses the following name mangling procedure: Names of groupings and typedefs appearing at the top level of the YANG module hierarchy are prefixed with the module name and two underscore characters ("__"). Names of other groupings and typedefs, i.e., those that do not appear at the top level of a YANG module, are prefixed with their full path that consisting of two underscore characters followed by the names of all ancestor data nodes separated by double underscore. For example, consider the following YANG module which imports the standard module "inet-types" :

The resulting RELAX NG schema will then contain the following named patterns:

[aeiouy]* ... ... ]]>

When a YANG grouping is used, it may be modified in place either by a refinement or an augment. Refinements (see , Section 7.12.2) are specified as substatements of the corresponding 'uses' statement. The augment mechanism (, Section 7.15) is slightly different in that the 'augment' statement is not a substatement of 'uses' - in most cases it is its sibling. While this can be in principle used for augmenting a subtree that is specified in the same place (without a grouping), in virtually all cases the 'augment' statement is really useful only in connection with 'uses'. [Editor's note: It is hoped that syntax of both the refinement and augment will be unified in the future versions of YANG and, in particular, that the relation of 'augment' and 'usus' statements will be made explicit.] In order to correctly map a 'uses' statement with an refinement or augment, the translated contents of the grouping must be expanded and the augmenting or refinement nodes added to the subtree. The current syntax of 'augment' causes the additional complication related to the fact that it may not be immediately clear which 'uses' statement is the one to be augmented. To resolve this uncertainty, the mapping algorithm must find the corresponding groupings for all 'uses' statements in the local scope and compare their top-level nodes with the XPath expression in the argument of 'augment'. For example, consider the grouping

First, assume it is used without a refinement or augment:

Then the corresponding RELAX NG snippet is an exact analogy of YANG:

]]>

On the other hand, if the "cask-grp" grouping is used with a refinement or augment, for example:

Then the grouping is expanded and augmented in place:

... ... ]]>

This section describes in each of its subsections the mapping procedure for one YANG statement, as it is currently implemented in the YANG->DSDL translator plug-in for the pyang tool. It is a work in progress and the implementation is open for diFscussion and changes. [Editor's Note: May need to discuss the difference between mapping and pseudo code.] In accord with the description of the mapping algorithm in , we assume the following context: The data structure (or object) representing the YANG statement being mapped is stored in the "stmt" variable. In particular, "stmt.arg" is the argument of this statement. A pointer to the RELAX NG element that will become parent of the new XML fragment representing the translation of the current YANG statement is also available. Schematically, the context in the RELAX NG tree looks like this:

... next new schema stuff comes here ... ]]>

When we say here that a new element is inserted, it means that it becomes a child of <rng:parent_element>. When we say that <rng:new_element> is inserted and becomes the new parent element, the situation changes as follows:

... next new schema stuff comes here ... ]]>

Throughout this section, we use qualified names for all XML elements. The mapping of prefixes to namespace URIs is shown in the table below. Prefix Namespace URI ahttp://relaxng.org/ns/compatibility/annotations/1.0 dc http://purl.org/dc/terms dsrl http://purl.oclc.org/dsdl/dsrl dml urn:ietf:params:xml:ns:netmod:dsdl-attrib:1 sch http://purl.oclc.org/dsdl/schematron In order to avoid unnecessary complications, this section describes the mapping algorithm for single-file output, i.e., one of the alternatives in .

At the first occurrence of this statement, the following pattern definition is added to the RELAX NG schema (cf. , p. 172):

]]>

Reference to this pattern is then inserted for all 'anyxml' statements.

This statement MAY be ignored if by an implementation or mode that doesn't map YANG extensions, see . Otherwise, the argument value MUST be used as the name of the extension argument. It can be represented either as an XML element or attribute, depending on the "yin-element" statement. See also .

There are three cases that require different treatment: This statement MUST be ignored if it appears at the top level of a YANG module and adds nodes to a foreign module or submodule, since this additions have no impact on the YANG module where the statement appears. If the nodes are to be added to a schema subtree that is present in its entirety in the current context, the new nodes are inserted directly in the RELAX NG tree. If the subtree to be augmented (or part of it), results from an expansion of a grouping via the "uses" statement, the grouping must be expanded first and then the new nodes added. discusses the mapping of the "augment" statement in more detail.

Insert Dublin Core metadata term <dc:isPartOf>.

Handled within the "bits" type, see .

Insert <rng:group> element and handle all substatements.

Insert <rng:choice> element and handle all substatements.

Attach annotation @dml:config to the parent element and use stmt.arg as its value.

Insert Dublin Core metadata term <dc:contributor> and use stmt.arg as its content.

Using the procedure outlined in , determine whether the container is optional, and if so, insert the <rng:optional> element and make it the new parent element. Then insert <rng:element> element at the current position and use stmt.arg as the value of its @name attribute. Then handle all substatements.

Insert element <dsrl:default-content> with stmt.arg as its content.

If the statement is at the module top level, insert Dublin Core metadata term <dc:description> and use stmt.arg as its content. Otherwise, insert DTD compatibility element <a:documentation>. In order to get properly formatted in the RELAX NG compact syntax, this element must be inserted as the first child of the parent element.

Insert <rng:value> element and use stmt.arg as its content. Then handle the 'status' substatement. Other substatements ('value', 'description' and 'reference') are ignored because the <rng:value> element cannot contain foreign elements, see , Section 6.

This statement is only handled as a substatement of "must", see .

Handled only within the "must" statement, see . Ignored in other contexts.

Not implemented.

Insert <rng:define> element and set the value of its @name attribute to stmt.arg with prepended full path, i.e., names of all ancestor nodes in the YANG schema tree separated by double underlines - see . Then handle all substatements.

Ignored, as it is assumed that all imported modules are processed by the YANG parser and made available to the mapping algorithm.

Insert <rng:include> and set the value of its @href attribute to stmt.arg concatenated with the file extension ".rng".

Not implemented.

Attach attribute @dml:key to the parent element and use stmt.arg as its value.

Unless there is the "mandatory true;" substatement or unless an enclosing list declares this leaf among its keys, insert <rng:optional> element and make it the new parent element. Then insert <rng:element> element and use stmt.arg as the value of its @name attribute. Then handle all substatements.

If there is a 'min-elements' substatement with argument value greater than zero, insert <rng:oneOrMore> and make it the new parent element. Otherwise insert <rng:zeroOrMore> and make it the new parent element. Then insert <rng:element> and use stmt.arg as its value. If there is a 'min-elements' substatement with argument value greater than zero, attach attribute @dml:min-elements to the <rng:element> element and use the argument of the 'min-elements' statement as its value. If there is a 'max-elements' substatement, attach attribute @dml:max-elements to the <rng:element> element and use the argument of the 'max-elements' statement as its value. If there is an 'ordered-by' substatement, attach attribute @dml:ordered-by to the <rng:element > element and use the argument of the 'ordered-by' statement as its value. Then handle all remaining substatements.

Handled within the "string" type, see .

Handled exactly as the 'leaf-list' statement, see .

Handled within the 'leaf' statement, see .

Handled within 'leaf-list' or 'list' statements, see .

Handled within 'leaf-list' or 'list' statements, see .

This statement is not specifically handled. The mapping algorithm starts with its substatements.

Insert <sch:assert> element. The value of its @test attribute is set to stmt.arg in which all occurrences of the string "$this" are replaced by "." (single dot). If there is an 'error-message' substatement, its argument value is used as the content of <sch:assert>, otherwise the <sch:assert> element is empty.

Attach @ns attribute to the <rng:grammar> element (the root of the RELAX NG schema) and use stmt.arg as its value.

Not implemented.

Handled within 'leaf-list' and 'list' statements, see .

Insert Dublin Core metadata term <dc:creator> and use stmt.arg as its content.

Not implemented.

Handled within "keyref" type, see .

Handled within "string" type, see .

Not implemented.

Not implemented. In RELAX NG, the elements of the target namespace are unprefixed.

Handled within 'container' statement, see and also .

Handled within numeric types, see .

If this statement is at module top level, insert Dublin Core metadata term <dc:BibliographicResource> and use stmt.arg as its content. Otherwise insert <a:documentation> and use stmt.arg as its content. Within parent element children, insert it as the last of <a:documentation> elements (i.e., after translations of 'description' statements), but before any other subelements.

Insert Dublin Core metadata term <dc:issued> and use stmt.arg as its content.

Not implemented.

Attach attribute @dml:status to the parent element and use stmt.arg as its value.

This statement is not specifically handled. The mapping algorithm starts with its substatements.

References to derived types are handled in the same way as references to groupings via the 'uses' statement (): <rng:ref> element is inserted with a properly mangled name and definitions of derived types are copied from external modules as necessary. Most YANG built-in types have an equivalent in the XSD datatype library as shown in . YANG type XSD type Meaning int8byte8-bit integer value int16short16-bit integer value int32int32-bit integer value int64long64-bit integer value uint8unsignedByte8-bit unsigned integer value uint16unsignedShort16-bit unsigned integer value uint32unsignedInt32-bit unsigned integer value uint64unsignedLong64-bit unsigned integer value float32float32-bit IEEE floating-point value float64double64-bit IEEE floating-point value stringstringcharacter string booleanboolean"true" or "false" binarybase64Binarybinary data in base64 encoding Details about the mapping of individual YANG built-in types are given in the following subsections.

Insert empty <rng:empty> element.

These two built-in types do not allow any restrictions and are mapped simply by inserting <rng:data> element whose @type attribute is set to stmt.arg mapped according to .

This YANG built-in type has no equivalent in the XSD datatype library and is mapped to the RELAX NG "string" type:

]]>

Insert <rng:list> element and insert under it for each 'bit' substatement the following XML fragment:

bit_name ]]>

where bit_name is the name of the bit as found in the argument of the corresponding 'bit' statement.

Insert <rng:choice> and handle all substatements.

As there is no suitable counterpart for the YANG built-in "keyref" type in the XSD datatype library, this type is mapped to "string" by inserting <rng:data> element with @type attribute set to "string". In addition, <sch:assert> element is inserted as child of <rng:data,> which checks that a 'list' entry with the corresponding value of the key exists.

YANG built-in numeric types are "int8", "int16", "int32", "int64", "uint8", "uint16", "uint32", "uint64", "float32" and "float64". They are handled by inserting <rng:data> element with @type attribute set to stmt.arg mapped according to . All numeric types support the 'range' restriction, which is handled in the following way: If the range expression consists of a single range part, insert the pair of RELAX NG facets

...]]>

and

...]]>

Their contents are the lower and upper bound of the range part, respectively. If the range part consists of a single number, both "minInclusive" and "maxInclusive" facets use this value as their content. If the lower bound is "min", the "minInclusive" facet is omitted and if the upper bound is "max", the "maxInclusive" facet is omitted. If the range expression has multiple parts separated by "|", then repeat the <rng:data> element once for every range part and wrap them all in <rng:choice> element. Inside each <rng:data> element, the corresponding range part is handled as described in the previous item. For example, the 'typedef' statement

typedef rt { type int32 { range "-6378..0|42|100..max"; } }

translates to the following RELAX NG fragment:

-6378 0 42 42 100 ]]>

This type is mapped by inserting the <rng:data> element with the @type attribute set to "string". For the 'pattern' restriction, insert <rng:param> element with @name attribute set to "pattern". The argument of the 'pattern' statement (regular expression) becomes the content of this element. The 'length' restriction is handled in the same way as the 'range' restriction for the numeric types, with the additional twist that if the length expression has multiple parts, the "pattern" facet

...]]>

if there is any, must be repeated inside each copy of the <rng:data> element, i.e., for each length part.

Handled exactly as the 'grouping' statement, see .

Insert <sch:assert> element. Its test checks that no two list items (sibling elements) have the same combination of values of leafs listed in stmt.arg. The content of the <sch:assert> element, i.e., the error message, is formed by a concatenation of "Not unique: " and stmt.arg. This works in conjunction with the dml:unique annotation.

Attach attribute @dml:units to the parent element and use stmt.arg as its value.

Check whether the grouping that the statement refers to is defined in the same module that is being translated. If it is so, go to step . [the grouping is defined in another module] If the same grouping has been already used in the module that is being translated, go to step . [first use of an external grouping] Copy the grouping definition from the external module and install its translation according to with a properly mangled name (see ). Insert <rng:ref> element and set its @name attribute to stmt.arg after performing the name mangling procedure described in , taking into account whether the grouping is local or external. Handling of substatements, and in particular the refinement statements, is not implemented yet.

Not implemented.

Not implemented.

Not implemented. Its stmt.arg may be checked by the mapping algorithm in order to make sure that the module is compatible.

Not implemented.

Simple Network Management Protocol (SNMP) Research Performance Systems International Performance Systems International Massachusetts Institute of Technology (MIT), Laboratory for Computer Science DCMI ISO/IEC ISO/IEC ISO/IEC ISO/IEC Cisco Systems, Inc. SNMPinfo TU Braunschweig

> dsrl:default-content ["config"] } access-strings = ( "read" | "write" | "create" | "delete" | "execute" ) mustUse-flag = element mustUse { xsd:boolean >> dsrl:default-content ["false"] } manual-validation = element manual-validation-rule { string-with-lang } # Semantic hints list-order = element order { ("any-order" | "user-order") >> dsrl:default-content ["any-order"] } container-existence = element existence { empty } units = element units { xsd:string { pattern="[^: \n\r\t]+" } # allow familiar units, but no whitespace or absolute URIs here | xsd:anyURI { pattern="([a-zA-Z][a-zA-Z0-9\-\+\.]*:|\.\./|\./|#).*" } # allow absolute URIs, plus relative URIs with ./ or ../ # prohibit relative URIs that could look like a unit, ex: m/s } # Definition copied from definition of gml:Uomidentifier from # Section 8.2.3.6 of Geography Markup Language (GML) v3.2.1 string-with-lang = ( attribute xml:lang { xsd:language }, xsd:string ) conformStatus = element status { "active" | "deprecated" | "obsolete" >> dsrl:default-content ["active"] } # the mustUnderstand element contains space separated list of namespace # tokens that need to be supported to correctly process the data model mustUnderstand = element mustUnderstand { list { xsd:NCName } } dml-netconf-error-app-tag = attribute netconf-error-app-tag { xsd:string } dml-phase-attribute = attribute phase { "fragment" | "std" | "full" } dml-moduleDefault = attribute moduleDefault { xsd:boolean >> dsrl:default-content ["false"] } ]]>

This appendix demonstrates output of the YANG->DSDL mapping algorithm applied to the "canonical" DHCP tutorial data model. It is presented as a single-file annotated RELAX NG schema (output alternative in ). shows the result of the mapping algorithm in the RELAX NG XML syntax and the same in the compact syntax, which was obtained using the Trang tool. The long regular expressions for IP addresses etc. would exceed the limit of 72 characters per line, so they were replaced by a dummy text. Other than that, the results of the automatic translations were not changed.

yang-central.org Partial data model for DHCP, based on the config of the ISC DHCP reference implementation. YANG module 'dhcp' (automatic translation) configuration and operational parameters for a DHCP server. 7200 The default-lease-time must be less than max-lease-time 600 false ethernet token-ring fddi ... IPv4 address regexp ... ... IPv6 address regexp ... ... date-and-time regexp ... A reusable list of subnets Allows BOOTP clients to get addresses in this range Options in the DHCP protocol See: RFC 2132, sec. 3.8 See: RFC 2132, sec. 3.17 7200 ... IPv4 prefix regexp ... ... IPv6 prefix regexp ... ([\p{L}\p{N}]+\.)*[\p{L}\p{N}] ]]>

> dsrl:default-content [ "7200" ] }?, [ dml:units = "seconds" ] element default-lease-time { xsd:unsignedInt >> sch:assert [ test = ". <= ../max-lease-time" "The default-lease-time must be less than max-lease-time" ] >> dsrl:default-content [ "600" ] }?, __subnet-list, element shared-networks { [ dml:key = "name" ] element shared-network { element name { xsd:string }, __subnet-list }* }?, element status { [ dml:key = "address" ] element leases { element address { inet-types__ip-address }, element starts { yang-types__date-and-time }?, element ends { yang-types__date-and-time }?, element hardware { element type { "ethernet" | "token-ring" | "fddi" }?, element address { yang-types__phys-address }? }? }* }? >> dml:config [ "false" ] }? inet-types__ip-address = inet-types__ipv4-address | inet-types__ipv6-address inet-types__ipv4-address = xsd:string { pattern = "... IPv4 address regexp ..." } inet-types__ipv6-address = xsd:string { pattern = "... IPv6 address regexp ..." } yang-types__date-and-time = xsd:string { pattern = "... date-and-time regexp ..." } yang-types__phys-address = xsd:string ## A reusable list of subnets __subnet-list = [ dml:key = "net" ] element subnet { element net { inet-types__ip-prefix }, element range { ## Allows BOOTP clients to get addresses in this range element dynamic-bootp { empty }?, element low { inet-types__ip-address }, element high { inet-types__ip-address } }, ## Options in the DHCP protocol element dhcp-options { [ dml:ordered-by = "user" ] ( ## See: RFC 2132, sec. 3.8 element router { inet-types__host }*), ## See: RFC 2132, sec. 3.17 element domain-name { inet-types__domain-name }? }?, [ dml:units = "seconds" ] element max-lease-time { xsd:unsignedInt >> dsrl:default-content [ "7200" ] }? }* inet-types__ip-prefix = inet-types__ipv4-prefix | inet-types__ipv6-prefix inet-types__ipv4-prefix = xsd:string { pattern = "... IPv4 prefix regexp ..." } inet-types__ipv6-prefix = xsd:string { pattern = "... IPv6 prefix regexp ..." } inet-types__host = inet-types__ip-address | inet-types__domain-name inet-types__domain-name = xsd:string { pattern = "([\p{L}\p{N}]+\.)*[\p{L}\p{N}]" } ]]>