Friday, 29 January 2016 15:15

Pcap vs Pcapng : File Information and Convert

Written by 
Rate this item
(0 votes)

Pcap and Pcapng : Determining the difference in the capture file and converting it

Exploring magic numbers, capinfos, hexdump, editcap and wireshark

Finding out what type of capture file you have, what the differences are and converting them to utilize different tools for analysis.  I've been capturing files using a number of tools recently (scapy, tcpdump, wireshark, dumpcap, etc) and once I began using different methods to analyse my capture files, I began learning and understanding about pcap and pcapng files formats and their differences.

Although I use a linux platform in all my examples detailed below,  the information provided can still be of assistance in exploring and analysing capture files.

 

Determining if the file is Pcap vs Pcapng

Depending on what you use to create the captures, the version of libpcap used, weither or not a single or multiple interfaces were selected during the capture all play a part in what format the capture file will be.

It appears the simplest way to determine what format the capture file is, is by looking at the header signature of the file or also more specifically the Section Header Blocks.  The global file header is called the magic number by some in forensically determining different file formats.  This byte sequence can vary in length, but regarding pcap and pcapng files, we are interested in the first 4 bytes.  A pcapng file can contain more than one section header block in a single file, and a more in depth hex analysis of the file can certainly provide other details, but the focus of this experiment is only determining the differences between the two formats.  Regarding pcap and pcapng files, the first 4 bytes of the capture file will determine what format the file is as well as byte ordering in pcap files (byte ordering in pcapng is determined in bytes 0x09-0x0B of the section header blocks).

 

The first 4 bytes are: (prefix 0x denotes that the number is in Hex)

pcap*: 0xD4C3B2A1

pcapng: 0x0A0D0D0A

*As mentioned earlier, the pcap 4 bytes 0xD4C3B2A1 (little-endian) can be in the reverse order as well 0xA1B2C3D4 (big-endian), depending on the byte order of the capture.  The byte order or little-endian/big-endian is determed by the capture machine and how it stores its information in memory.  Little-endian where the least significant byte is stored in the least significant position, vs Big-endian has the most significant byte in the least significant position.

 

All of my pcap files start with 0xD4C3B2A1, but I have come across some documents that state where some other versions (possibly older) can have different bytes such as 0x34CDB2A1.  I personally have not seen this, but I'm sure there could be other variations.  Pcapng however, according to the internet-draft for the pcapng standard, the first 4 bytes will always be 0x0A0D0D0A

 

- Capinfos

We can use a tool that is part of the wireshark suite called Capinfos.  Capinfos can tell you different information about the capture file including the type (which we are interested in)

capinfos -t <capture-file>
  • -t : flag specifies the type

Here is a look at a pcap file I captured earlier today

pcap pcapng 01

Here's the same file after it's been converted to pcapng (using wireshark)

pcap pcapng 02

 

I wanted to see if I manually change the first 4 bytes of the header in a pcap file, how capinfos would respond.  In my hex editor I simply replaced 0xD4C3B2A1 with 0x0A0D0D0A in the first 4 bytes

pcap pcapng 05

and ran capinfos again on the standard pcap file

pcap pcapng 06

So, apparently it looks at more than just the first 4 bytes to determine what type of file it is (good to know!).  Wireshark was unable to open this file either (it is corrupted at this point)

 

- Hexdump

Seeing the actual bytes can help in certain circumstances.  Using a hex editor or simply hexdump in the linux command line can save some time.

hexdump -C -n 4 <capture-file>
  • -C : flag specifies Canonical hex+ASCII display
  • -n : specifies number of bytes

Here we see the same pcap file from before

pcap pcapng 03

and pcapng

pcap pcapng 04

 

Differences between Pcap and Pcapng files

There are some major differences in the physical file structure between the two formats.  Starting with the header as we saw earlier, and conitnuing on throughout the file.  Pcap-new generation, can include multple interfaces, embedded and appended data, and a more strict standard than the initial pcap format.

- From https://www.winpcap.org/ntar/draft/PCAP-DumpFileFormat.html#formatidb

  • Extensibility: aside of some common functionalities, third parties should be able to enrich the information embedded in the file with proprietary extensions, which will be ignored by tools that are not able to understand them.
  • Portability: a capture trace must contain all the information needed to read data independently from network, hardware and operating system of the machine that made the capture.
  • Merge/Append data: it should be possible to add data at the end of a given file, and the resulting file must still be readable.

 

Converting Pcap <> Pcapng

There are a few ways to convert between the two file formats.  When a single interface is used in the capture, and as long as the files are not corrupt, it's easy to convert between the two.

- Wireshark

You can open up either a pcap or pcapng file in WIreshark.  Click File > Save As... and select the File type: you wish to save it in

pcap pcapng 09

 

- Editcap

Pcap > Pcapng

editcap -F pcapng <input-pcap-file> <output-pcapng-file>

Pcapng > Pcap

editcap -F pcap <input-pcapng-file> <output-pcap-file>

 

Converting pcapng > pcap when multple interfaces are present

Pcapng supports using multple interfaces in a single capture file.  Pcap does not.  Wireshark or editcap will not allow you to convert to a pcap format when multple interfaces are present in the capture file.

pcap pcapng 10

 

- Wireshark

In these cases you must split the file into multiple files for each interface present.  Once this is done, editcap still will not convert the pcapng file to pcap as the frame interface id is still present in the capture.  The simplest way is to use wireshark, which is a 2 step process.

Split the files up using the display filter frame.interface_id==x, where x= the interface you want to extract.  File > Export Specified Packets and save to a pcapng file.  You will then need to open up the new saved pcapng file and File > Save As and select pcap as your new format.

pcap pcapng 11 

 

- pcapng.com

You can also use the site http://pcapng.com/ for pcapng files up to 8MB, even with multple interfaces, the site will split up the files for you.

 

When all else fails

I came across an interesting post, https://medium.com/@dan.ellis/solving-a-tricky-problem-d1a8766f5347#.bk9se4nhy where he had the need to manually edit the interface id in a pcapng file.  You can see in one of the pictures of his post, that the first 4 bytes of the header are 0x0A0D0D0A, which tells us that indeed it has a pcapng file signature.  He ended up needing to change the interface id (from 0x01 to 0x00) inside the Enhanced Packet Blocks 0x00000006 as somehow it was missing the second Interface Description Block which would be for interface id 0x01.  Once he "labelled" all of the interface ids in the Enhanced Packet Blocks to 0x00, which would now reference the first Interface Description Block, it resolved his issue and allowed wireshark to open up the file,   It's just goes to show you that if you're determined, there maybe other ways of looking at capture files even if they are corrupt, in order to find what you are searching for.

I have also come across some other utilities that may assist with corrupt packet captures.

  • Bless: (or any other hex editor of your liking)
  • pcapfix: pcapfix tries to repair your broken pcap and pcapng files
  • editcap: part of the wireshark suite, can perform a variety of manipulation tasks

 

Just for curiosity, I created a similar corrupt file by running his code "backwards" and replacing my interface id's of 0x00 to 0x01, and ran pcapfix against the file to see if it would find and fix the problem.

pcap pcapng 07

It did find that the Interface Description Block was missing, but when I opened up the fixed file in wireshark, although it opened, it was processing the file differently.  It appears because pcapfix added another IDB, wireshark looks at the information as if there were 2 interfaces capturing the data??? 

pcap pcapng 08 

 

References:

http://www.garykessler.net/library/file_sigs.html

http://www.kroosec.com/2012/10/a-look-at-pcap-file-format.html

http://stackoverflow.com/questions/15222047/weird-pcap-header-of-byte-sequence-0a-0d-0d-0a-created-on-mac

https://ask.wireshark.org/questions/24916/how-to-know-if-a-file-without-extension-contains-pcap-or-pcap-ng-formatted-data

https://www.winpcap.org/ntar/draft/PCAP-DumpFileFormat.html

 

 

 

Read 6247 times Last modified on Tuesday, 02 February 2016 19:25
Algis Salys

Creator and owner of algissalys.com.  Linux enthusiast, electronics tinkerer, and likes to spend time in the workshop building and creating new projects.