OpenTTCN/Knowledge base/Standard codec explained

From OpenTTCN

Standard Codec Explained

This article describes work in progress. Information on this page can change as the implementation evolves. If you want to be posted on the most recent news on this topic, please contact us at support@openttcn.fi.

This article discusses usage and encoding format of the standard codec shipped with OpenTTCN SDK for C#, OpenTTCN SDK for C++, found in OpenTTCN Tester 2011 4.1.4 and later. Programming examples of its use are also provided, as well as example encodings.

The standard codec component is available for C# and C++ and is released as a technical preview in OpenTTCN Tester 2011 4.1.4. Users of OpenTTCN Tester are encouraged to provide their feedback on the subject through support@openttcn.fi.

Overview of the standard codec

The standard codec allows you to obtain:

• bit-level encoding with generally no octet alignment for values of useful and common ETSI-specified types defined in:

• UsefulTypes module (see ETSI ES 201 873-1 V4.2.1 (2010-07) Annex E (informative): Library of Useful Types);

• LibCommon_BasicTypesAndValues, LibCommon_DataStrings, and LibCommon_TextStrings modules found, for example, in the LTE IMS test suite (ETSI STF160).

• octet-aligned encoding for values of arbitrary constrained and unconstrained types, including:

• basic types: integer, bitstring, octetstring, boolean, charstring, universal charstring, hexstring, verdicttype, float;

• constructed types: enumerated, union, record, record of, set, set of.

Decoding is performed against a hypothesis (TciType) which shall be made available to the decoder in order for decoding to succeed.

Values of useful and common ETSI-specified types are encoded and decoded according to available encode and variant attributes attached to a type.

Values of arbitrary constrained and unconstrained types are processed to provide compact and efficient encoding, yet not to make the overall encoding rules too complex, what sometimes leads to suboptimal encoding, still close to optimal, but with the benefit of having simpler encoder and decoder.

Supported variant and encode attributes

Supported variant and encode attributes are:

• variant attributes for integer and float types from the UsefulTypes module, except "IEEE754 extended float" and "IEEE754 extended double";
• variant attributes for integer and boolean types from the LibCommon_BasicTypesAndValues module;
• encode attributes for bitstring and octetstring types from the LibCommon_DataStrings module, except length range;
• length encode attributes for charstring types from the LibCommon_TextStrings, except length range.

Octet-aligned integer types are encoded in the network byte order, i.e. most significant octet first. Non octet-aligned integers are encoded in the most significant bit first order.

Float types are encoded as IEEE754 double by default, unless "IEEE754 float" variant attribute is present, in which case value is encoded as IEEE754 float. In both cases bytes are arranged in the network byte order.

Examples of encodings are presented in a separate section of this article.

Current version of the standard codec

Current version of the standard codec is 00.00.01.001. Explanations in this article are applicable to this particular version unless otherwise stated.

You can check which version of the standard codec you are using programmatically.

// Create a new instance of the latest available version of the standard codec:

ICodecComponent cd = OpenTTCN.Sdk.StartHereCD.
    GetCodecComponent("http://www.openttcn.com/Codecs/2011/Standard");

// Obtain version of the created instance:

string version = cd.ComponentVersion;

To request a specific version of the codec component with the specified name, see GetCodecComponent() overload in the SDK for C# API reference.

// Create a new instance of the latest available version of the standard codec:

OpenTTCN::SDK::CodecComponent* cd = OpenTTCN::SDK::StartHereCD::
    getCodecComponent("http://www.openttcn.com/Codecs/2011/Standard");

// Obtain version of the created instance:

std::string version = cd->getComponentVersion();

To request a specific version of the codec component with the specified name, see getCodecComponent() overload in the SDK for C++ API reference.

Invoking encode and decode methods

Encoding example:

using OpenTTCN.Sdk;

using Etsi.Ttcn3.Tci;
using Etsi.Ttcn3.Tri;

...

// Create a new instance of the standard codec:

ICodecComponent cd = StartHereCD.GetCodecComponent("http://www.openttcn.com/Codecs/2011/Standard");

ITciValue value;

// Obtain the ITciValue somehow:
...

ITriMessage enc = cd.Encode(value);

// The ITriMessage contains the resulting value encoding.

Decoding example:

// Create a new instance of the standard codec:

ICodecComponent cd = StartHereCD.GetCodecComponent("http://www.openttcn.com/Codecs/2011/Standard");

ITriMessage enc;
ITciType t;

// Obtain the ITriMessage somehow, together with the type of the message to be put into ITciType:
...

ITciValue value = cd.Decode(enc, t);

// The ITciValue contains the resulting decoded value.

The above encode and decode examples in C# work for arbitrarily complex values of basic and constructed types.

You may call Encode() and Decode() methods on the same instance multiple times.

The Encode() and Decode() methods are not thread-safe. You need to create multiple instances of the codec if you plan to invoke Encode() and Decode() methods concurrently.

Encoding example:

#include <StartHereCD.h>
#include <CodecComponent.h>
#include <tci/TciValues.h>

#include <memory>

using namespace OpenTTCN::SDK;

using namespace ORG_ETSI_TTCN3_TCI;
using namespace ORG_ETSI_TTCN3_TRI;

...

// Create a new instance of the standard codec:

std::auto_ptr<OpenTTCN::SDK::CodecComponent> cd(StartHereCD::
    getCodecComponent("http://www.openttcn.com/Codecs/2011/Standard"));

TciValue* value;

// Obtain the TciValue somehow:
...

TriMessage* enc = cd->encode(value);

// The TriMessage contains the resulting value encoding.

Decoding example:

// Create a new instance of the standard codec:

std::auto_ptr<OpenTTCN::SDK::CodecComponent> cd(StartHereCD::
    getCodecComponent("http://www.openttcn.com/Codecs/2011/Standard"));

TriMessage* enc;
TciType* t;

// Obtain the TriMessage somehow, together with the type of the message to be put into TciType:
...

TciValue* value = cd->decode(enc, t);

// The TciValue contains the resulting decoded value.

The above encode and decode examples in C++ work for arbitrarily complex values of basic and constructed types.

You may call encode() and decode() methods on the same instance multiple times.

The encode() and decode() methods are not thread-safe. You need to create multiple instances of the codec if you plan to invoke encode() and decode() methods concurrently.

Example of custom codec

Here we show an example of a simple custom codec built atop of the standard codec component.

The custom codec uses the standard codec as a delegate to perform all core coding tasks, then wraps the result of the encoding into an envelope adding TTCN-3 type name to the encoding header, what makes the custom encoder and decoder fully self-contained.

• CD_impl.cs

Note that the example above is not thread-safe. For thread safety you need to create a local copy of the delegate codec in Encode() and Decode(), or synchronize their critical sections with a mutex.

• CD_impl.cpp
• CD_impl.h

Note that the example above is not thread-safe. For thread safety you need to create a local copy of the delegate codec in encode() and decode(), or synchronize their critical sections with a mutex.

Encoding rules and examples of encoding

Integer

Constrained

Integers constrained by the variant attribute to carry N bits are encoded with bit-level precision, most significant bit first. Exactly N bits are added to the encode buffer.

Constrained integers can be signed or unsigned. Sign expansion is performed for signed integers during decoding. No sign expansion is performed for unsigned integers during decoding.

Constrained integers are generally non octet-aligned.

Octet-aligned

See Table 1: Examples of constrained octet-aligned integer encodings.

Non octet-aligned

See Table 2: Examples of constrained non octet-aligned integer encodings.

Unconstrained

Arbitrary unconstrained integer is encoded as follows:

• 1 octet containing:

• 1 bit sign: 0 for non-negative integers, 1 for negative integers; followed by 7 bits representing unsigned integer length determinant N that is a minimum number of octets necessary to fit all non-zero bits of integer bit representation if the integer is non-negative, or all non-one bits of integer bit representation if the integer is negative;

• if the specified integer value to be encoded is zero or minus one, N is 1;

• the above octet representing sign (1 bit) and length N in octets (7 bits) is followed by N octets carrying integer value representation in network byte order, i.e. most significant octet first.

See Table 3: Examples of unconstrained integer encodings.

Boolean

Constrained boolean is encoded into the number of bits specified in the variant attribute with high-order bits set to 0 and the lowest-order bit carrying actual boolean value, 1 for true and 0 for false. Constrained boolean is generally non octet-aligned.

Unconstrained boolean is encoded into 1 octet with high-order bits set to 0 and the lowest-order bit carrying actual boolean value, 1 for true and 0 for false.

See Table 4: Examples of boolean encodings.

Bitstring

Constrained bitstring is encoded into the number of bits specified in the encode attribute. Bits are added to the encode buffer in the order of their declaration in the bitstring value, from left to right. Constrained bitstring is generally non octet-aligned. If effective length of the bitstring value being encoded is greater than the number of bits specified in the encode attribute, say N, the value is truncated on the right and only the leftmost N bits make their way into the resulting encoding. If effective value length is less than the length specified in the encode attribute, runtime exception is thrown, causing a test case error.

Unconstrained bitstring is encoded as a length determinant representing effective bitstring length in bits followed by the payload consisting of the minimum number of octets (possibly zero) necessary to fit the bitstring.

The payload contains bit-level bitstring representation possibly followed by pad zero bits on the right to make the payload octet-aligned.

If bitstring length is zero, no payload is present.

Length determinant itself is encoded as an unconstrained integer.

Starting with version 00.00.01.002 of the standard codec, the following clause applies:

Value of bitstring type constrained with encode "length(x..y)" kind of encode attribute is encoded into the number of bits contained in the effective bitstring value, with no extra length determinant as per value field. A convention is used during decoding that the field of one of the allowed basic data types immediately preceding the value in question is understood as an integer length determinant that determines the length of the value to be decoded in bits.

Allowed basic data types for such dedicated length determinant field are: integer, octetstring, hexstring, and bitstring.

See Table 5: Examples of bitstring encodings.

Octetstring

Constrained octetstring is encoded into the number of octets specified in the encode attribute. Octets are added to the encode buffer in the order of their declaration in the octetstring value, from left to right. If effective length of the octetstring value being encoded is greater than the number of octets specified in the encode attribute, say N, the value is truncated on the right and only the leftmost N octets make their way into the resulting encoding. If effective value length is less than the length specified in the encode attribute, runtime exception is thrown, causing a test case error.

Unconstrained octetstring is encoded as a length determinant representing effective octetstring length in octets followed by the payload consisting of the minimum number of octets (possibly zero) necessary to fit the octetstring.

The payload contains bit-level octetstring representation.

If octetstring length is zero, no payload is present.

Length determinant itself is encoded as an unconstrained integer.

Starting with version 00.00.01.002 of the standard codec, the following clause applies:

Value of octetstring type constrained with encode "length(x..y)" kind of encode attribute is encoded into the number of octets contained in the effective octetstring value, one octet per two hexadecimal digits, with no extra length determinant as per value field. A convention is used during decoding that the field of one of the allowed basic data types immediately preceding the value in question is understood as an integer length determinant that determines the length of the value to be decoded in octets.

Allowed basic data types for such dedicated length determinant field are: integer, octetstring, hexstring, and bitstring.

Charstring

Constrained charstring is encoded into the number of octets specified in the encode attribute, one octet per character, where each octet is carrying ISO/IEC 646 character encoding in its lower 7 bits, with the high order bit set to zero. Characters are added to the encode buffer in the order of their declaration in the charstring value, from left to right. If effective length of the charstring value being encoded is greater than the number of characters specified in the encode attribute, say N, the value is truncated on the right and only the leftmost N characters make their way into the resulting encoding. If effective value length is less than the length specified in the encode attribute, runtime exception is thrown, causing a test case error.

Unconstrained charstring is encoded as a length determinant representing effective charstring length in characters followed by the payload consisting of the minimum number of octets (possibly zero) necessary to fit the charstring, one octet per character.

If charstring length is zero, no payload is present.

Length determinant itself is encoded as an unconstrained integer.

Starting with version 00.00.01.002 of the standard codec, the following clause applies:

Value of charstring type constrained with encode "length(x..y)" kind of encode attribute is encoded into the number of octets equal to the number of characters contained in the effective charstring value, one octet per character, with no extra length determinant as per value field. A convention is used during decoding that the field of one of the allowed basic data types immediately preceding the value in question is understood as an integer length determinant that determines the length of the value to be decoded in characters.

Allowed basic data types for such dedicated length determinant field are: integer, octetstring, hexstring, and bitstring.

Universal charstring

Universal charstring is encoded as a length determinant representing effective length of universal charstring UTF-8 encoding in octets followed by the payload consisting of the minimum number of octets (possibly zero) necessary to fit the UTF-8 encoding of universal charstring.

The payload contains UTF-8 encoding of the universal charstring.

If universal charstring length is zero, no payload is present.

Length determinant itself is encoded as an unconstrained integer.

Starting with version 00.00.01.002 of the standard codec, the following clause applies:

Value of universal charstring type constrained with encode "length(x..y)" kind of encode attribute is encoded into the number of octets necessary to fit its UTF-8 representation, with no extra length determinant as per value field. A convention is used during decoding that the field of one of the allowed basic data types immediately preceding the value in question is understood as an integer length determinant that determines the length of the UTF-8 representation of the value to be decoded in octets.

Allowed basic data types for such dedicated length determinant field are: integer, octetstring, hexstring, and bitstring.

Float

Float is encoded as IEEE754 double with octets arranged in network byte order, unless "IEEE754 float" variant attribute is attached to the float subtype, in which case float is encoded as IEEE754 float with octets arranged in network byte order.

Hexstring

Constrained hexstring is encoded into the number of nibbles, or 4-bit groups, specified in the encode attribute. 4-bit encodings of hex digits are added to the encode buffer in the order of hex digits declaration in the hexstring value, from left to right. Constrained hexstring is generally non octet-aligned. If effective length of the hexstring value being encoded is greater than the number of nibbles specified in the encode attribute, say N, the value is truncated on the right and only the leftmost N nibbles make their way into the resulting encoding. If effective value length is less than the length specified in the encode attribute, runtime exception is thrown, causing a test case error.

Unconstrained hexstring is encoded as a length determinant representing effective hexstring length in hex digits followed by the payload consisting of the minimum number of octets (possibly zero) necessary to fit the hexstring.

The payload contains bit-level hexstring representation, 4 bits per hex digit, possibly followed by pad zero bits on the right to make the payload octet-aligned.

If hexstring length is zero, no payload is present.

Length determinant itself is encoded as an unconstrained integer.

Starting with version 00.00.01.002 of the standard codec, the following clause applies:

Value of hexstring type constrained with encode "length(x..y)" kind of encode attribute is encoded into the number of nibbles (4-bit entities) contained in the effective hexstring value, one nibble per hexadecimal digit, with no extra length determinant as per value field. A convention is used during decoding that the field of one of the allowed basic data types immediately preceding the value in question is understood as an integer length determinant that determines the length of the value to be decoded in nibbles.

Allowed basic data types for such dedicated length determinant field are: integer, octetstring, hexstring, and bitstring.

Enumerated

Unconstrained enumerated value is encoded as an unconstrained integer value representing default encoding of the enumeration item as deduced from the corresponding TTCN-3 or ASN.1 type specification.

Starting with OpenTTCN Tester 4.2.0beta3.0 (C#) and OpenTTCN Tester 4.2.0beta4.0 (C++), the following clause applies:

Constrained enumerated value is encoded into the number of bits specified in the encode attribute. Bits are added from left to right, most significant bit first. Constrained enumerated value is generally non octet-aligned.

type enumerated
CustomEnumType
{
    EnumOne     (1),
    EnumTen    (10),
    EnumTwenty (20)
};

type enumerated
CustomEnumType2
{
    EnumOne     (1),
    EnumTen    (10),
    EnumTwenty (20)
}
with
{
    encode "length(8)"
}

Verdicttype

Verdict value is encoded as 1 octet representing unsigned integer carrying TCI-compliant integer code of the verdict value in its low-order bit part.

Union

Union is encoded as an unconstrained integer value representing zero-based index of the selected variant in the variant list as defined by the corresponding TTCN-3 or ASN.1 type specification, followed by the encoding of the selected variant value.

type union CustomUnionType
{
    integer number,
    charstring string,
    CustomEnumType enum
};

Record

Record value is encoded as a header part followed by encodings of non-omitted fields added to the encode buffer in the order of field declarations in the record type. Omitted fields produce no encoding in the output, except zero presence bit indicator in the header, as described below.

The header part consists of the minimum number of octets (possibly zero) necessary to fit presence bit indicators for optional fields.

For each optional record field there is a presence bit indicator in the generated encoding, set to 1 if the field is present and 0 if it is omitted, followed by padding zero bits if the generated bit sequence is not octet-aligned. Presence bit indicators are ordered according to ordering of optional fields in the record type definition.

The header is missing (of zero length) if there are no optional fields defined in the record.

Encoding of empty record with no optional fields is empty (header + fields), i.e. it occupies zero octets.

type record SimpleRecordType
{
    integer numberField,
    charstring stringField optional,
    octetstring octetField optional
}

{
    numberField := -1024,
    stringField := omit,
    octetField := omit
}

{
    numberField := -1024,
    stringField := "ABC",
    octetField := omit
}

{
    numberField := -1024,
    stringField := "ABC",
    octetField := 'CFE5'O
}

{
    numberField := -1024,
    stringField := omit,
    octetField := 'CFE5'O
}

Record of

Record of value is encoded as a length determinant representing effective number of non-omitted elements in the record of array, followed by encodings of non-omitted elements added to the encode buffer in the order of their occurrence in the container record of value.

Length determinant itself is encoded as an unconstrained integer.

type record of charstring
CharstringList;

Set

Set is encoded as a length determinant representing effective number of non-omitted fields in the set value, followed by a sequence of index-value pairs added to the encoding buffer in the order of presence of non-omitted set fields in the set value container.

Omitted set fields are ignored, and their encoding is not added to the encode buffer.

Length determinant is encoded as an unconstrained integer.

Each index-value pair represents encoding of a zero-based index of a particular field name in the field name list as defined for the corresponding set type, followed by the corresponding non-omitted field value encoding.

Each field name index is encoded as an unconstrained integer.

type set SimpleSetType
{
    integer numberField,
    charstring stringField optional,
    octetstring octetField optional
}

Set of

Encoding of set of is identical to encoding of record of.

type set of charstring
CharstringSet;