Yeah the labels are loaded into the Detectron2 format from the 3D annotations json at train time, we plan to add similar data loading for YOLOv3 etc soon. Starting out with Detectron2 was mainly for POC/demo purposes, the idea in the future is to be able to feed the data anywhere it might be needed.