Navigating CCS Charging Session File Formats
Some Background
In 2022, QualityLogic set out to build a product to accelerate interoperability between EV charging stations and EVs, using the popular Combined Charging Systems (CCS). The focus of our research and development was to analyze the results of actual charging sessions so we could quickly identify the causes of charging failures that were the result of interoperability issues with the CCS implementations.
Rather than duplicating existing tools that were focused on capturing and displaying the charging session interactions, allowing an expert engineer could determine the cause of a failure, our focus was on automating the analysis of the captured files of the charging sessions. Our goal was to speed up the identification of “root” causes and other interoperability issues that are so frustrating to EV owners trying to charge their EVs at a fast-charging station.
At first, we looked at adopting or building our own traffic capturing system. However, we quickly realized that the numerous tools already on the market could export captured charging session message traffic in a standard message packet capture format. This file format, known as PCAP or PCAPNG (PCAP Next Generation), is used in numerous domains that rely on information communicated in discreet “message packets” and can be dissected and viewed in the open-source Wireshark application.
Standard file formats, such as PCAP, offer significant value for interoperability testing. It enables the engineering teams to readily exchange such files and use readily available tools for analyzing them. Wireshark has been the predominant tool to date for viewing and analyzing PCAP files because of its versatility and openness. The QualityLogic CCS Analyzer provides organizations with another valuable reason to standardize on PCAP files for interoperability analysis.
However, in the process of bringing our CCS Analyzer to the community, we discovered that while most tools can produce PCAP files, not everyone has such tools or understands how to generate PCAP files from them.
The purpose of this article is to share some of the file format issues we have encountered in tackling the challenge of analyzing charging sessions for interoperability issues. Fortunately, most of the format challenges we have seen can be remedied by simply converting other file formats to a standard PCAP file.
More Than One File Format
During development and early use of the CCS Analyzer, we have encountered different types of file formats and we’ve been asked to analyze them as an alternative to available PCAP files. In addition, we have noticed inconsistencies between capture systems – e.g., some include pulse-width modulation (PWM) data while others do not, or some include a TLS Private Key while others do not.
Binary Logging Format
Other than standard PCAP or PCAPNG files, the first file format we encounter is the Binary Logging Format (BLF) which is a Vector proprietary format, the primary storage format for their logger hardware. This is what many of the EV manufacturers use in the field. Vector provides utilities to convert from BLF to PCAP, as does Wireshark itself.
HomePlug
The second type of file format request has been to either analyze native HomePlug PLC logs or convert the proprietary logging formats from HomePlug chip vendors into a PCAP format. We have not found any utilities that perform these conversions, but technically it shouldn’t be difficult. And while analyzing native log formats is certainly feasible, the lack of standardization between chip vendors makes this a more challenging and expensive task.
ASCII Log File Format
The third type of file format is ASCII Log File Format. It is simply a text file containing all of the information in the message packets. Vector has their own format which they refer to as ASC, but many other vendors may put their logging data into a raw text format, although not necessarily formatted the same as Vector.
There are certainly other file formats that are used to capture messaging traffic, but these three are the most common ones we have encountered in the EV charging domain so far. Our position is that the most efficient solution for the community is to standardize on the PCAP or PCAPNG formats to enable scaling of the analysis of interoperability charging sessions industry-wide.
There are many nuances when converting from one format to another. Our expertise in this area enables us to help customers understand if there are specific types of interoperability anomaly checks that couldn’t be run with their proprietary files converted to PCAP files.
We also help customers convert their file formats to PCAP files so they can be analyzed by our CCS Analyzer. For instance, a major EV OEM was interested in using our Analyzer, but their captured traffic was in the Vector BLF file format. We were able to point them at a Wireshark utility that converted BLF to PCAP files which could then be successfully analyzed with our Analyzer. We plan to add automatic conversion to PCAPs in our CCS Analyzer when it encounters BLF files.
The following sections explore the major file formats used in the CCS community for capturing interoperability message traffic.
PCAP/PCAPNG
PCAP / PCAPNG Logging Files
- Purpose: Widely used for capturing network traffic at the packet level.
- Tools: Wireshark, tcpdump.
- Details: Contains raw packet data and metadata such as timestamps, protocols, and payload information.
- Message-based format for reading and writing
- Packet Capture Next Generation Dump File Format (binary format)
- Standard format for storing captured data send over a network
- PCAPNG Supports Ethernet and AFDX, PCAP supports only Ethernet
- All other bus systems, system variables, internal events, …are not supported
- See also: Difference between PCAP, PCAPNG and CANoe Specific BLF Logging
Vector BLF
BLF Logging Format (BLF)
- Purpose: Used in CAN (Controller Area Network) bus logging.
- Tools: Vector CANalyzer, CANape.
- Details: Efficient binary format optimized for automotive data.
- Message-based format for reading and writing
- Binary Logging Format (binary format)
- Stores the data in binary format in a very efficient way in terms of file size and read/write performance
- Supports messages of all bus systems, system variables, environment variables, internal events, markers, and comments
- BLF is recommended for all new loggings using Vector capture/logging equipment
Automotive-Specific Utilities for BLF
- Vector Tools
- Tools:
- CANalyzer/CANape: Convert between BLF, ASC, and CSV.
- VXCONVERT: Standalone tool for converting Vector log files.
- Supported Formats: BLF <-> ASC, BLF/ASC <-> CSV.
- Tools:
Proprietary Logging Formats
HomePlug chip vendors do not adhere to a universal standard log format. Instead, they often implement proprietary logging formats tailored to their specific chipsets, firmware, and development tools. These formats are generally used to record debug information, data exchanges, and events within HomePlug networks.
Common Characteristics of HomePlug Vendor Logs:
- Proprietary Formats:
- Vendors like Qualcomm Atheros, Broadcom, or Marvell typically use unique formats for logs.
- Logs often require vendor-specific tools or software for decoding.
- Common Log Types:
- PHY Layer Logs: Logs raw physical layer data such as signal-to-noise ratio (SNR) or power levels.
- MAC Layer Logs: Record traffic exchanges, protocol information, and device connectivity states.
- Network Diagnostics: Logs to diagnose HomePlug AV/AV2 or G.hn performance and network issues.
- Formats Used:
- Binary Logs: Compact, raw data logs for use with proprietary debugging tools.
- Text-Based Logs: More human-readable but often tailored for specific debugging outputs.
- CSV/JSON/XML: Occasionally provided for broader compatibility or when logs are exported via diagnostic tools.
Tools Provided by Vendors:
- Qualcomm (Atheros) Powerline Toolkit:
- Includes logging capabilities for HomePlug AV/AV2 chipsets.
- Logs often require specific decoding libraries included in their SDKs.
- Marvell G.hn Tools:
- Provide logs in proprietary formats for performance analysis.
- May include options to export logs to text or CSV.
- Broadcom Diagnostic Tools: LogToCSV/OpenLogViewer
- Purpose: Convert proprietary log formats into human-readable CSV.
- Supported Formats: Generic text, CSV, or user-defined formats.
- Custom Tools
- Many companies develop custom scripts or utilities for internal use based on Python, MATLAB, or Java to handle proprietary formats.
- Typically vendor-specific binary logs that must be parsed using their software.
With the adoption of G.hn as a universal standard for wired home networking, some vendors are moving towards more standardized formats, but HomePlug-specific formats remain vendor-specific.
ASCII Logging File Format
- ASC (ASCII Logging Format):
- Purpose: Human-readable alternative to BLF for CAN bus logging.
- Tools: Vector tools.
- Details: Easier to read but less efficient than BLF.
- ASCII is not recommended at high data rates due to its poor performance. BLF and MDF format are much better suited for this.
File Formats and Conversions
Common Features in Log Formats
- Timestamping: Accurate to microseconds or nanoseconds for synchronization.
- Protocol-Specific Parsing: Can parse protocol headers and payloads.
- Compression: Binary formats like BLF or PCAP support efficient data storage.
- Extensibility: Formats like JSON or XML allow metadata tagging.
Considerations When Converting File Formats
- Data Loss: Some formats (e.g., binary logs) may store metadata (timestamps, identifiers) that could be lost in conversion to simpler formats (e.g., CSV).
- Protocol Decoding: Ensure the tool can parse protocol-specific data like CAN signals (requires DBC files) or IEC 61850 fields.
- Performance: Binary formats like BLF and PCAP are much faster to process and store than plain text or CSV.
If the vendor does not provide a clear mechanism for reading logs, you can try the following:
- Vendor SDKs/Tools:
- Most vendors supply tools to read and convert proprietary logs.
- Look for documentation in the chipset’s SDK or development kit.
- Custom Scripts:
- Use reverse engineering (if allowed under license agreements) to create custom parsers for binary logs.
- For text logs, Python libraries like pandas can be helpful for processing.
- Wireshark:
- HomePlug AV/AV2 traffic can sometimes be captured via PCAP files using a powerline adapter with diagnostic capabilities.
- Wireshark may provide partial decoding if dissectors are available.
Conclusions
Interoperability of fast-charging between EVs and EVSEs is a critical success factor in accelerating EV adoption globally. The CCS community is investing significant resources in testing interoperability, identifying the issues that prevent successful charging and negotiating solutions when issues are discovered.
While we believe that solving the interoperability challenges for CCS requires rigorous conformance testing and certification, today’s model for interoperability is to simply test an EV with an EVSE, and then fix any issues on a bilateral basis.
QualityLogic has spent the last three years investigating interoperability issues and developing tools to accelerate the process of finding interoperability anomalies. In the process, we have come to understand that one key interoperability issue is the standardization of messaging traffic captured file formats. We hope this blog provided you useful insight into what we have learned and why we recommend standardizing on the PCAP/PCAPNG file format in order to make analysis of interoperability charging sessions more efficient and more effective.
Are you interested in learning more about QualityLogic’s CCS Analyzer? You can download our most recent data sheet, or if you’re interested in a free trial or more information, please let us know using the form below.