Octet string in UPER encoding of DSRCData.ApplicationList shifted 2 bits to the left

[rmwesley]

First off, I love this repo!
And secondly, I believe this is not really an Issue, rather a question.
I think raising Issues for my questions probably annoys the maintainers.
Therefore, could someone enable Discussions for this repo?

Anyway, about my question...

ITS ISO 14906 and EN 15509 for EFC / DSRC use PER (Packed Encoding Rules). EN ISO 12813 for CCC / DSRC uses it as well.
But the encoding of an ApplicationList exemple I built using the Aligned and Unaligned PER codecs of pycrate do not match the examples provided in Annex B of the first 2 norms when I tried them.

Here is a small test example for encoding a BST and an ApplicationList (containing an EFC-CM):

from pycrate_asn1dir.ITS_r1318 import DSRCData

BST = DSRCData.BST
utc_ts = 1103790512

bst_value = {
  'rsu': {
    'manufacturerid': 0x1,
    'individualid': 1052 #41C
    },
  'time':utc_ts,
  'profile': 0,
  'mandApplications': [
    {
    'aid': 1
    }
    ],
  'profileList': []
  }
BST.set_val(bst_value)

print(f"Expected PER encoded BST: 800041C41CA81B000010100")
print(f"BST encoded in UPER in hex: {BST.to_uper().hex().upper()}")
print()

contract_provider = "30C002"
toc = 0x0001
cv = 0x2
efc_cm = f"{contract_provider}{toc:04X}{cv:02X}"
print(f"EFC-CM: {efc_cm}")

AttributeList = DSRCData.AttributeList
AttributeList.set_val([{
  'attributeId': 0,
  'attributeValue': ('octetstring', bytes.fromhex(efc_cm))
  }
])
#print(f"AttributeList:", AttributeList.to_asn1())
#print(f"AttributeList in JSON:", AttributeList.to_jer())

print(f"Expected PER encoded AttributeList: 01000206{efc_cm}")
print("AttributeId 0 is for EFC-CM, which is of type OCTET STRING (2), AND OF SIZE 6")
print("AttributeList encoded in UPER in hex:", AttributeList.to_uper().hex().upper())
`

And here is the output:

Expected PER encoded BST: 800041C41CA81B000010100
BST encoded in UPER in hex: 0000800041C41CA81B0000101000

EFC-CM: 30C002000102
Expected PER encoded AttributeList: 0100020630C002000102
AttributeId 0 is for EFC-CM, which is of type OCTET STRING (2), AND OF SIZE 6
AttributeList encoded in UPER in hex: 01000818C30008000408

So the BST encoding is OK up to padding!

But for some reason, in the UPER encoding of the AttributeList, the octet string of length 6 (containing the EFC-CM) is shifted 2 bits to the left.
I also tried out the APER codec, but to no avail...

Does anyone have any idea what I may be doing wrong?

[mitshell]

Could you please provide a reference for the specs you are mentioning? Or maybe they are not available publicly (eventually send them to me by email)? It will be hard to help if I don't have access to the comparison element.
On the other side, the module https://github.com/pycrate-org/pycrate/tree/master/pycrate_asn1dir/ETSI_ITS_r1318 was taken from the ETSI forge 7 years ago iirc, and maybe outdated. Some structures may have changed since then. That could explain the discrepancies.

[mitshell]

And thank you for the introductory message. It's OK ask questions in issues: as I am currently the only active maintainer / developer for the project, I prefer to keep the project management as simple as possible.

[rmwesley]

Thanks for the answer, I was thinking the same!
I think somewhere in the Wiki is is mentioned ETSI_ITS_r1318 is old. I will try to gather a good set of ASN.1 specifications to compile for France (EFC) and then maybe Germany (CCC). Once I do it for France (TIS-PL) I will give an update here.
I wish there was some online resource where I could find the bundle of ASN.1 DSRC specs used by France easily. Also for each European country.
Maybe it is simple to find and I am just not looking at the right place haha.

Sadly using the most recent version of a standard is not always the "most appropriate" choice. A device can use/implement an older version of a norm and still be accepted in most countries. And the most recent specification usually is not immediately widely adopted.

Also, I work for a TSP so I have a few of the norms, but I don't think they are openly available.
The ISO ASN.1 specs are, though:
https://standards.iso.org/iso/14906/
https://standards.iso.org/iso/12813/

If you look in the webpage for the ISO 12813 (CCC), it states CCC is suitable for the Italian DSRC (UNI), and its norm is openly available:
ETSI ES 200 674-1 V2.4.1 (2013-05)

That is the only norm I know that can easily be found for free.
Still, the Italian DSRC is the biggest exception to the rule. It does not use an EFC (AID=1) or CCC (AID=20) application. It uses a different application entirely, called UNI (with AID=29).

[rmwesley]

You were right! I just had to compile the ASN1 specs and use them properly. Sorry for wasting your time.
I just compiled the EfcDsrcApplicationv9.1, EfcDsrcGenericv10.1 and EfcDataDictionary V1.5 ASN specs to efc.py.
It also works if ISO12813 is added (CCC).

I was hesitant to use the most recent specs fearing they may not be backwards compatible with the 2011 version of ISO 14906. But all the UPER encodings I tested match with the informative examples provided in the ISO 14906 (2011) PDF document !

So here is a code snippet with some tests I did for whoever is interested:

from efc import EfcDsrcGeneric, EfcDataDictionary
#from ccc import EfcDsrcGeneric, EfcDataDictionary
#from pycrate_asn1dir.ITS_CCC import EfcDsrcGeneric, EfcDataDictionary

BeaconID = EfcDsrcGeneric.BeaconID
BeaconID.set_val({
  'manufacturerid': 0x1,
  'individualid': 1052 #41C
  })
print(BeaconID.to_asn1())
print("BeaconID UPER:", BeaconID.to_uper().hex().upper())
BST = EfcDsrcGeneric.BST
utc_ts = 1103790512

bst_value = {
  'rsu': {
    'manufacturerid': 0x1,
    'individualid': 1052 #41C
    },
  'time':utc_ts,
  'profile': 0,
  'mandApplications': [
    {
    'aid': 3
    }
    ],
  'profileList': []
  }
BST.set_val(bst_value)

print(f"BST encoded in UPER in hex: {BST.to_uper().hex().upper()}")
print()

contract_provider = "30C001"
toc = 0x0001
cv = 0x02
efc_cm_str = f"{contract_provider}{toc:04X}{cv:02X}"
print(f"EFC-CM: {efc_cm_str}")
EfcContextMark = EfcDataDictionary.EfcContextMark
EfcContextMark.from_uper(bytes.fromhex(f"{efc_cm_str}"))
print("EFC-CM from UPER encoding in JER:", EfcContextMark.to_jer())
efc_cm = {
  'contractProvider': {
    'countryCode': (195, 10),
    'providerIdentifier': 1
    },
  'typeOfContract': b'\x00\x01',
  'contextVersion': 2
  }

EfcContextMark.set_val(efc_cm)
print(EfcContextMark.to_asn1())
print("EFC-CM in UPER encoding:", EfcContextMark.to_uper().hex().upper())

EfcContainer = EfcDsrcGeneric.EfcContainer
EfcContainer.set_val(('efccontext', EfcContextMark._val))
print(EfcContainer.to_asn1())

EfcContainer.set_val(('attrList', [
  {
  'attributeId': 0,
  'attributeValue': ('octetstring', bytes.fromhex(efc_cm_str))
  }
]))
print("EFC Container with AttrList encoded in UPER in hex:", EfcContainer.to_uper().hex().upper())
print(EfcContainer.to_asn1())

And here is the output after execution:

{
  manufacturerid 1,
  individualid 1052
}
BeaconID UPER: 000100008380
BST encoded in UPER in hex: 0000800041C41CA81B0000103000

EFC-CM: 30C001000102
EFC-CM from UPER encoding in JER: {
 "contextVersion": 2,
 "contractProvider": {
  "countryCode": "30c0",
  "providerIdentifier": 1
 },
 "typeOfContract": "0001"
}
{
  contractProvider {
    countryCode '0011000011'B,
    providerIdentifier 1
  },
  typeOfContract '0001'H,
  contextVersion 2
}
EFC-CM in UPER encoding: 30C001000102
efccontext : {
  contractProvider {
    countryCode '0011000011'B,
    providerIdentifier 1
  },
  typeOfContract '0001'H,
  contextVersion 2
}
EFC Container with AttrList encoded in UPER in hex: 090100020630C001000102
attrList : {
  {
    attributeId 0,
    attributeValue octetstring : '30C001000102'H
  }
}

9 is the preamble/choice index for AttributeList for the EfcContainer CHOICE.
There are no tags in PER:
https://www.oss.com/asn1/resources/asn1-made-simple/asn1-quick-reference/packed-encoding-rules.html

Also AttributeList is a Parameterized type, so I learned I simply need to use a container to contain it. I lost a lot of time recompiling ASN1 specs thinking I was doing something wrong lol.

I will do some tests with some actual DSRC devices and report back here to share whatever info I get!

[mitshell]

Thanks a lot for the feedback. Do not hesitate to update this issue with any new findings.

[rmwesley]

I can't believe how easy to use this package is after understanding containers!! Parameterized types should just be put inside a container to contain them and everything works!

In the EfcDsrcGeneric v10.1 ASN there is a Container called T-APDU. I am switching request with all the devices using it!

Here is my follow-up. I have this data structure in Python:

from efc import EfcDsrcGeneric, EfcDataDictionary

get_resp_t_apdu_bytes = b't\x04\x01 @\x0f\x0f\x0f\x0f\x0f\x0f\x0f\x0f\x0f\x0fY_\x00\x00'
print(f"T-APDU containing Get-Response encoded in UPER in hex: {get_resp_t_apdu_bytes.hex().upper()}")
EfcDsrcGeneric.T_APDUs.from_uper(get_resp_t_apdu_bytes)

print(f"Python T-APDU value:\n{EfcDsrcGeneric.T_APDUs._val}")
print(f"T-APDU encoded in JER: {EfcDsrcGeneric.T_APDUs.to_jer()}")

Which outputs the following:

Python T-APDU value:
('get-response', {'fill': (0, 1), 'eid': 4, 'attributelist': [{'attributeId': 32, 'attributeValue': ('paymeans', {'personalAccountNumber': b'\x0f\x0f\x0f\x0f\x0f\x0f\x0f\x0f\x0f\x0f', 'paymentMeansExpiryDate': {'year': 2034, 'month': 10, 'day': 31}, 'paymentMeansUsageControl': b'\x00\x00'})}]})
T-APDU encoded in JER: {
 "get-response": {
  "attributelist": [
   {
    "attributeId": 32,
    "attributeValue": {
     "paymeans": {
      "paymentMeansExpiryDate": {
       "day": 31,
       "month": 10,
       "year": 2034
      },
      "paymentMeansUsageControl": "0000",
      "personalAccountNumber": "0f0f0f0f0f0f0f0f0f0f"
     }
    }
   }
  ],
  "eid": 4,
  "fill": "00"
 }
}

I love the .to_jer() and .to_jval() methods. They are great for interfacing with JSON APIs and also good for logging!

AttributeList makes up a list of lists since they are defined like so in the ASN:
AttributeList{Container} ::= SEQUENCE (SIZE(0..127,...)) OF Attributes{Container}

Attributes{Container} ::= SEQUENCE {
attributeId INTEGER (0..127,...),
attributeValue Container
}

So I have to manually write a key-value mapping for the attributes. Which is easy in Python with dict comprehension, of course!

[rmwesley]

One question, though. I already know how to add ASN specs to pycrate_asn1dir/ and compile them after updating the dicts in pycrate_asn1c/specdir.py. I simply put the DSRC, EFC, CCC and LAC ASNs into a folder called ETSI_ITS_EFC and then added a 'ITS_EFC': 'ETSI_ITS_EFC' key-value pair to the ASN_SPECS_ITS dict.

My question is: what should I call this ASN spec bundle instead of the "ETSI_ITS_EFC" string I made up?
I wonder if there is some defined ASN bundle name or something for EFC/tolling.

I found a few details on arc-it:
https://www.arc-it.net/html/standards/standard154.html
https://www.arc-it.net/html/standards/standard254.html

There are no "bundles" for these standards on arc-it.net.
And not any entries on for ISOs 12813 and 13141, which rely a lot on the DSRC Application Layer and EFC specifications.

[mitshell]

These ITS ASN.1 specs seem really to have spreaded, being reused and adapted in several countries and by multiple SDOs. I can't answer your question btw, choose the naming which you think is closest to the SDO in charge and domain addressed.