ISO 15118-2 Timeout Behavior Analysis
QualityLogic strongly believes that conformance to specification is a critical prerequisite for achieving interoperability in message exchange protocols. However, in ecosystems where there is no mechanism to enforce conformance testing, interoperability with non-conformant implementations often becomes a business necessity.
The Challenge of Interoperability Testing
Discovering and implementing workarounds for non-conformant systems can be a time-consuming and painful process. This is particularly true in the EV charging space, specifically with respect to the ISO 15118 message exchange protocol, where interoperability problems can result in charge failures and very unhappy customers.
But the current process used for interoperability testing produces GB of test session traffic that contains valuable data that is typically not used to solve these problems because of the time required to analyze and compile the data.
This article explores a methodology for anticipating potential interoperability problems by analyzing a historical sample of captured communication traffic. The goal was to determine the frequency distribution of protocol characteristics relative to their conformance limits, and to determine if that data showed a compelling case for allow some out of specification traffic to sustain interoperability with other implementations.
Building the CCS Analyzer
During the development of QualityLogic’s CCS Analyzer product, we accumulated 242 charge-session traffic samples from 70 different sources, including charger and vehicle manufacturers, industry interoperability testing events, open-source projects, test labs, and other contributors. These files were originally used to enhance the CCS Analyzer’s ability to identify anomalies in message-exchange traffic that could lead to interoperability issues. We later realized that the same data could be further analyzed to examine the frequency distribution of various protocol behaviors.
With some minor adjustments, we were able to adapt the CCS Analyzer to quickly extract data for most of the timeout and numerical parameters defined in the ISO 15118 specifications. Timeouts in which the protocol state machine explicitly mandates session termination for non-conformant behavior proved to be an ideal focus for this statistical look at protocol behavior.
Findings and Analysis
Data for 25 timeout characteristics was extracted from each of the 242 captured traffic files and stored in a large JSON dataset. This dataset was then analyzed by an AI engine using a robust series of prompt guiding the AI to extract the data desired. The table below summarizes the scope of the analysis and provides some details on the out of specification behavior in the dataset.
| Timer Name | Sources | PCAP Files | Measurements | Out of Spec | % |
|---|---|---|---|---|---|
| V2G_EVCC_Authorization_Timer | 78 | 225 | 285 | 0 | 0 |
| V2G_EVCC_CableCheck_Timer | 58 | 132 | 154 | 3 | 1.9 |
| V2G_EVCC_ChargeParameterDiscovery_Timer) | 76 | 200 | 246 | 0 | 0 |
| V2G_EVCC_Msg_Timer(AuthorizationReq) | 78 | 225 | 8145 | 1 | 0 |
| V2G_EVCC_Msg_Timer(CableCheckReq) | 58 | 132 | 9916 | 1 | 0 |
| V2G_EVCC_Msg_Timer(CertificateInstallationReq) | 2 | 24 | 24 | 1 | 4.2 |
| V2G_EVCC_Msg_Timer(CertificateUpdateReq) | 2 | 2 | 2 | 1 | 50 |
| V2G_EVCC_Msg_Timer(ChargeParameterDiscoveryReq) | 76 | 200 | 880 | 1 | 0.1 |
| V2G_EVCC_Msg_Timer(ChargingStatusReq) | 19 | 50 | 9589 | 1 | 0 |
| V2G_EVCC_Msg_Timer(CurrentDemandReq) | 55 | 125 | 138785 | 807 | 0.6 |
| V2G_EVCC_Msg_Timer(MeteringReceiptReq) | 4 | 28 | 90 | 1 | 1.1 |
| V2G_EVCC_Msg_Timer(PaymentDetailsReq) | 14 | 54 | 58 | 1 | 1.7 |
| V2G_EVCC_Msg_Timer(PaymentServiceSelectionReq) | 82 | 235 | 295 | 1 | 0.3 |
| V2G_EVCC_Msg_Timer(PowerDeliveryReq) | 67 | 179 | 376 | 1 | 0.3 |
| V2G_EVCC_Msg_Timer(PrechargeReq) | 55 | 125 | 1696 | 2 | 0.1 |
| V2G_EVCC_Msg_Timer(ServiceDetailReq) | 5 | 29 | 29 | 1 | 3.4 |
| V2G_EVCC_Msg_Timer(ServiceDiscoveryReq) | 82 | 236 | 296 | 1 | 0.3 |
| V2G_EVCC_Msg_Timer(SessionSetupReq) | 82 | 236 | 296 | 1 | 0.3 |
| V2G_EVCC_Msg_Timer(SessionStopReq) | 65 | 160 | 176 | 1 | 0.6 |
| V2G_EVCC_Msg_Timer(SupportedAppProtocolReq) | 81 | 234 | 282 | 1 | 0.4 |
| V2G_EVCC_Msg_Timer(WeldingDetectionReq) | 48 | 93 | 308 | 1 | 0.3 |
| V2G_EVCC_Ongoing_Timer | 79 | 227 | 828 | 8 | 1 |
| V2G_EVCC_PreCharge_Timer | 55 | 125 | 143 | 1 | 0.7 |
| V2G_SECC_Sequence_Timer | 84 | 241 | 170677 | 19 | 0 |
Of particular interest were a series of charts generated by the AI engine showing the frequency distribution of timing values and how they clustered relative to the conformance limits defined in both the ISO 15118-2 and ISO 15118-20 specifications. A few representative charts are shown in the following pages.
While some out-of-specification data samples were observed for various timing characteristics, they represented an extremely small portion of the overall data set. Furthermore, most in-specification samples fell comfortably within the conformance limits.
Our analysis concluded that the probability of timeout-related failures in this dataset was very low, and the data did not justify relaxing conformance limits to ensure interoperation with other implementations.
A caveat is that the analysis was done on a very small dataset of charging session PCAP files skewed towards sessions with charging issues.
That said, the analysis highlights the value of examining aggregated traffic patterns to make informed risk assessments when implementing a message exchange stack.
The process would be more valuable to analyze a larger dataset of charging sessions. This is easy to do with the CCS Analyzer. Then an AI engine could be trained to evaluate, flag patterns and even identify the probable causes and suggest fixes to them.
If you want to learn more about the CCS Analyzer and try it out or want more information about how we analyzed all these files, contact jmater@qualitylogic.com or info@qualitylogic.com.