Data format

The carlanomaly package reads all of this for you: it loads every modality as a PyTorch dataset, downloads the parts it needs and evaluates detectors at all four tiers. Install it with pip install git+https://github.com/carlanomaly/devkit; its documentation covers the full API. The rest of this page is for reading the files directly.

Directory structure

Scenarios are grouped by split and CARLA town (Town01–Town05, Town10HD). In the anomaly test split, scenarios are additionally grouped by anomaly type (e.g. change-weather, vanish-actor).

Inside a scenario

Every scenario-<n>/ contains the entries below. Files inside a scenario are numbered by timestep: rgb-front/000099.jpg is the front image at timestep 99. Tags show the download part that ships each entry and the splits it occurs in. Select a part to see what it adds.

Cameras

Every pixel carries a semantic class and an instance ID. Since these annotations were generated in a simulation, they are exact.

Segmentation

Segmentation masks are RGBA PNGs named by timestep, e.g. segmentation-front/000099.png. The R channel holds the semantic class ID, the G and B channels the instance ID:

import numpy as np
from PIL import Image

img = np.asarray(Image.open("segmentation-front/000099.png"))
class_ids = img[:, :, 0]
instance_ids = img[:, :, 1].astype(np.int32) | (img[:, :, 2].astype(np.int32) << 8)

Depth

Depth maps are 8-bit grayscale PNGs named by timestep, e.g. depth-front/000099.png, in CARLA’s logarithmic encoding. They hold the distance along the camera’s viewing axis and convert to metres as follows:

depth = np.asarray(Image.open("depth-front/000099.png"), dtype=np.float32)
metres = 1000 * np.exp(5.70378 * (depth / 255 - 1))

The encoding covers 3.3 m to 1000 m, and each grey level spans about 2 % of the distance.

Semantic LiDAR

Point clouds for each timestep with per-point semantic class and instance labels. Instance IDs match those in the camera segmentation masks.

Each file loads with pandas:

import pandas as pd
points = pd.read_feather("pointclouds/000000.feather")

KITTI annotations

Every camera has a kitti-<camera>/ directory with 3D bounding boxes of the objects in its view, in the KITTI object format: label_2/ holds one label file per timestep and calib/ the camera calibration. extended/ contains the same boxes as feather tables, together with the simulator’s attributes of each object.

Anomaly annotations

Anomaly labels are provided at four levels of granularity, matching the evaluation tiers.

Sample level

For cameras, anomaly-<camera>/ holds one single-channel PNG per timestep, where 0 means normal and any other value means anomaly.

For LiDAR, anomaly-lidar/ holds one feather file per timestep with an anomaly column, one row per point of the matching point cloud.

Sensor level

anomaly-<sensor>/sensor.feather holds one row per timestep with an anomaly column.

Timestep level

A timestep is anomalous when any sensor reading at that point in time contains an anomaly. anomaly-observation.feather holds one row per timestep:

Scenario level

Scenario labels are given by the directory: everything under test/anomaly/ is anomalous.

Additional data

Each scenario also contains the following readings, one row per timestep unless noted:

All of them load as pandas dataframes:

import pandas as pd
imu = pd.read_feather("imu.feather")