IP-ID: Classifying IP-ID host behavior in the wild

Originally used to assist network-layer fragmentation and reassembly, the IP identification field (IP-ID) has been used and abused for a range of tasks, from counting hosts behind NAT, to detect router aliases and, lately, to assist detection of censorship in the Internet at large. These inferences have been possible since, in the past, the IP-ID was mostly implemented as a simple packet counter: however, this behavior has been discouraged for security reasons and other policies, such as random values, have been suggested. These policies can be inferred by leveraging two vantage points sending packets that arrived interleaved at the target host: the resulting sequences (denoted as constant, local, global, random and “odd”) are illustrated in the following pictures (where the two vantage points are represented as a white box and a purple circle).

Technique

In this page, we collect and make available useful resources concerning our IP-ID classification technique [PAM-18a] , in particular:

• Measurement We propose a framework to classify the different IP-ID behaviors using active probing from a single host using IP spoofing to precisely control packet interleaving at the generation. Despite being only minimally intrusive, our technique is significantly accurate (99% true positive classification) robust against packet losses (up to 20%) and lightweight (few packets suffices to discriminate all IP-ID behaviors). We haven’t done the effort of cleaning up the code to release it to the public (contact us if interested).

• Classification Our technique is based on the definition of highly discriminative features with known expected values, that we train over a dataset with manually constructed ground truth, that we make available to the community. Notice that by employing a supervised technique, we are also able to cope with odd behaviors such as those due to load balancing middleboxes, or host implementations using the wrong endianness. The ground truth dataset we release can of course be useful to validate any other technique you may wish to define, so don’t hesitate to grab a copy!

Results

In this page, we also make available results concerning the application of our IP-ID classification [PAM-18a] technique to a broad Internet-wide census we perfomed in 2017, in particular:

• We actively probe one alive target per each routable /24 subnet: we find (see the following picture) that that the majority of hosts adopts a constant IP-IDs (39%) or local counter (34%), that the fraction of global counters (18%) significantly diminished, that a non marginal number of hosts have an odd behavior (7%) and that random IP-IDs are still an exception (2%). If you want to have more details (e.g., per /24 view) don’t hesitate to grab a copy of the census dataset!

Datasets

• Ground truth dataset
  • Ground truth dataset G denoted with G in [PAM-18a] (hold your breath, we’ll be activating the link shortly,…)
  • Ground truth dataset G’ an alternative ground truth dataset which we use to validate the generality of our findings (more details will come in a technical report).
• Census dataset
  • Large scale dataset denoted with L in [PAM-18a] from which the above results have been gathered

Publications

• [PAM-18a] Salutari, Flavia and Cicalese, Danilo and Rossi, Dario, A closer look at IP-ID behavior in the Wild International Conference on Passive and Active Network Measurement (PAM) Mar. 2018, conference Abstract Bibtex
  Originally used to assist network-layer fragmentation and reassembly, the IP identification field (IP-ID) has been used and abused for a range of tasks, from counting hosts behind NAT, to detect router aliases and, lately, to assist detection of censorship in the Internet at large. These inferences have been possible since, in the past, the IP- ID was mostly implemented as a simple packet counter: however, this behavior has been discouraged for security reasons and other policies, such as random values, have been suggested. In this study, we propose a framework to classify the different IP-ID behaviors using active probing from a single host. Despite being only minimally intrusive, our technique is significantly accurate (99% true positive classification) robust against packet losses (up to 20%) and lightweight (few packets suffices to discriminate all IP-ID behaviors). We then apply our technique to an Internet-wide census, where we actively probe one alive target per each routable /24 subnet: we find that that the majority of hosts adopts a constant IP-IDs (39%) or local counter (34%), that the fraction of global counters (18%) significantly diminished, that a non marginal number of hosts have an odd behavior (7%) and that random IP-IDs are still an exception (2%).

  @inproceedings{DR:PAM-18a,
    title = ,
    author = {Salutari, Flavia and Cicalese, Danilo and Rossi, Dario},
    booktitle = {International Conference on Passive and Active Network Measurement (PAM)},
    annote = {category=article state=unpublished project=newnet},
    note = {topic=ipid},
    address = {Berlin, Germany},
    year = {2018},
    month = mar,
    url = {http://www.enst.fr/~drossi/paper/rossi17pam18a.pdf},
    howpublished = {http://www.enst.fr/~drossi/paper/rossi17pam18a.pdf},
    www_txt_url = {http://www.enst.fr/~drossi/paper/rossi17pam18a.pdf}
  }