INSTRUCTIONS ABOUT HOW TO USE THE DATABASE Nightttime Vehicle Database (NVD) is a database aimed to test new algorithms for vehicle detection at night. For more details, and to download it, enter to the link provided below: - https://sites.google.com/view/visionlearning/databases/nighttime-vehicle-detection As it can be seen, the database is organized as follows: 1) "Images": there are five folders, one for each scenario: Richmond, Walnut, California, Conneticut and Gelderland. Inside each folder, there are two .zip files (considering "name_scenario" as the name of the scenario): name_scenario_train.zip and name_scenario_test.zip. Each .zip contains folders, one for each sequence, with a maximum of 1000 images each. In summary, the structure is as follows (with N training sequences and M test sequences belonging to the scenario "name_scenario" and considering #train_i# and #test_j# as indexes): - name_scenario -> name_scenario_train.zip * seq#train1# -> images seq#train1# * seq#train2# -> images seq#train2# ... * seq#trainN# -> images seq#trainN# -> name_scenario_test.zip * seq#test1# -> images seq#test1# * seq#test2# -> images seq#test2# ... * seq#testM# -> images seq#testM# 2) Folder "GT": there are five folders, one for each scenario: Richmond, Walnut, California, Conneticut and Gelderland. Inside each folder, there are two .zip files (considering "name_scenario" as the name of the scenario): name_scenario_train.zip and name_scenario_test.zip. Each .zip contains the ground-truth annotations, based on points, of the vehicles in the train and test sequences of each scenario. In summary, the folder corresponding to a scenario called "name_scenario" follows the structure described below (with N training sequences and M test sequences belonging to the scenario "name_scenario" and considering #train_i# and #test_j# as indexes): - name_scenario -> name_scenario_train.zip * groundTruth_seq_#train1#_vehicle.xml * groundTruth_seq_#train2#_vehicle.xml ... * groundTruth_seq_#trainN#_vehicle.xml -> name_scenario_test.zip * groundTruth_seq_#test1#_vehicle.xml * groundTruth_seq_#test2#_vehicle.xml ... * groundTruth_seq_#testM#_vehicle.xml Each .xml file also contains additional data about the sequence apart from the ground-truth annotations: the name of the scenario of the sequence, the number of sequence, the number of frames of the sequence and the image resolution (width and height). 3) File "code_gt_conversion.zip": This file contains several scripts written in Matlab, necessary to export ground-truth information represented in .xml to files compatible with the algorithms announced in the paper. In particular, the proposed system uses point-based annotations in .csv, while Faster-RCNN uses .json annotations with bounding boxes. The mentioned scripts will be explained in further detail after this section. The file "code_gt_conversion.zip" contains the following: - annotation2csv.m: this is a script which reads the ground-truth annotations in .xml files and exports the information to .csv files, containing point-based annotations of vehicles. These files have been used as input to the proposed foveatic system. - gtpoint2bbox.m: this script takes a .csv file containing point-based ground-truth, obtained from the script annotation2csv.m, and generates a bounding box for each point-based ground-truth location. Several bounding box sizes are proposed in the script. As a result, a .csv file is retrieved with ground-truth information in the form of bounding boxes. - GT_bbox_GT_mat.m: this script uses a .csv file, result from the script gtpoint2bbox.m, containing ground-truth information represented by bounding boxes and retrieves a .mat file with that ground-truth information and the images associated. - GT_mat_GT_JSON.m: this file takes as input a .mat file obtained from the script GT_bbox_GT_JSON.m and generates a .json file in the format required by Faster-RCNN, which needs bounding box annotations. - getFileSet.m: this contains a complementary function necessary to execute the rest of the scripts. How to use the scripts: - First, arrange the ground-truth data with the following structure (determine a folder with path "gt_path_root" where all ground-truth data will be contained): - gt_path_root -> Richmond * train -> .xml files of ground-truth data of the training sequences of Richmond * test -> .xml files of ground-truth data of the test sequences of Richmond -> Walnut * train -> .xml files of ground-truth data of the training sequences of Walnut * test -> .xml files of ground-truth data of the test sequences of Walnut ... - At the same time, arrange the images with the following structure (determine a folder with path "images_path_root" to store the images): - images_path_root - Images -> Richmond * train -> .xml files of ground-truth data of the training sequences of Richmond * test -> .xml files of ground-truth data of the test sequences of Richmond -> Walnut * train -> .xml files of ground-truth data of the training sequences of Walnut * test -> .xml files of ground-truth data of the test sequences of Walnut - To have the point-based annotations used by the proposed system based on foveal classifiers, the scripts annotation2csv.m and getFileSet.m are necessary. Only the script annotation2csv.m has to be executed for this purpose (this uses getFileSet.m which is a function). - The following scripts must be executed in the order shown below to obtain .json files which can be used by Faster-RCNN (the .json files are represented in the format specified in the COCO Dataset): annotation2csv.m -> gtpoint2bbox.m -> GT_bbox_GT_mat.m -> GT_mat_GT_JSON.m - How to use the script "annotation2csv.m": 1) Set the values of the following input parameters: "gt_xml_path_root": full path to the root directory where all .xml files will be read. If the ground-truth data is structured as shown above, this would correspond to "gt_path_root". For example: '/media/abn/Data1/Databases/NVD/GT' "gt_point_export_path": full path to the root directory where the .csv files, containing point-based annotations, will be stored. The same structure to arrange the output files as for the inputs, will be created, and this parameter can be the same path as stablished for "gt_xml_path_root" or it can be another. As examples: '/media/abn/Data1/Databases/NVD/GT' '/media/abn/Data1/Databases/NVD/GT_point_csv' 2) Run the script. 3) The output is composed of .csv files, one for each sequence, with the ground-truth annotations based in points of the vehicles contained in the sequence. Each file, with the name "groundTruth_seq_t_vehicle.csv", for the sequence number "t", has the following format: row i -> i, x1, y1, x2, y2, x3, y3, ..., xn, yn - Considering that: * n: number of vehicles in the sequence. * i: number of frame (from 1 to the number of frames of the sequence). * (xj, yj): the (x,y) pixel coordinates of the mass center of the jth vehicle in the frame "i". - How to use the script "gtpoint2bbox.m": 1) Set the values of the following input parameters: "num_seq": the number of sequence whose annotation will be taken as input. For NVD database, is a number from 1 to 135. "path_images_root": full path to the root directory where images are stored. If the images are stored with the structure shown above, this path would correspond to images_path_root. For example: '/media/abn/Data1/Databases/NVD/Images' "path_gt_root": full path to the root directory where the .csv files, containing point-based annotations, are stored. This would correspond to the parameter "gt_point_export_path" from "annotation2csv.m". As examples: '/media/abn/Data1/Databases/NVD/GT' '/media/abn/Data1/Databases/NVD/GT_point_csv' "gt_bbox_export_path": full path to the root directory where the .csv files, containing annotations based in bounding boxes, will be stored. The same structure to arrange the output files as for the inputs, will be created, and this parameter can be the same path as stablished for "path_gt_root" or it can be another. As examples: '/media/abn/Data1/Databases/NVD/GT' '/media/abn/Data1/Databases/NVD/GT_bbox_csv' 2) Run the script. 3) The output is a .csv file with the ground-truth annotations based in bounding boxes of the vehicles contained in the sequence. Several sizes are proposed in the script. Each file, with the name "groundTruth_seq_t_bbox_vehicle.csv", for the sequence number "t", has the following format: row i -> i, h1, w1, xinf1, yinf1, h2, w2, xinf2, yinf2, ... hn, wn, xinfn, yinfn - Considering that: * n: number of vehicles in the sequence. * i: number of frame (from 1 to the number of frames of the sequence). * (hj, wj, xinfj, yinfj): respectively, height, width and the coordinates of the upper-felt corner of the bounding box of the jth vehicle in the frame "i". - How to use the script "GT_bbox_GT_mat.m": 1) Set the values of the following input parameters: "num_seq": same as for "gtpoint2bbox.m". "path_images_root": same as for "gtpoint2bbox.m". "gt_bbox_path_root": full path to the root directory where the .csv files, containing annotations based in bounding boxes, are located. This would correspond to the parameter "gt_bbox_export_path" from "gtpoint2bbox.m". For instance: '/media/abn/Data1/Databases/NVD/GT' '/media/abn/Data1/Databases/NVD/GT_bbox_csv' "gt_mat_export_path": full path to the root directory annotations based in bounding boxes, will be stored in .mat files. The same structure to arrange the output files as for the inputs, will be created, and this parameter can be the same path as stablished for the previous paths to export ground-truth files, or it can be another. As examples: '/media/abn/Data1/Databases/NVD/GT_mat' '/media/abn/Data1/Databases/NVD/GT' 2) Run the script. 3) The output is a .mat file with the ground-truth annotations based in bounding boxes of the vehicles contained in the selected sequence. Each file, with the name "groundTruth_seq_t_vehicle.mat", for the sequence number "t", contains a Matlab struct with the following fields: row i -> n_frame, frame_name, type, id, bbox - Considering that: * n_frame: number of frame of the row i (from 1 to the number of frames of the sequence). * frame_name: the name of the image which is referred by n_frame. * type: a cell array with the string 'Vehicle' repeated as many times as the number of objects in the frame. * id: array with the ids of the vehicles in the frame. * bbox: array whose rows follow this structure: (xinfj, yinfj, wj, hj) respectively, the coordinates of the upper-felt corner and the width and height of the bounding box of the jth vehicle in the frame n_frame. - How to use the script "GT_mat_GT_json.m": 1) Set the values of the following input parameters: "num_seq": same as for "gtpoint2bbox.m". "path_images_root": same as for "gtpoint2bbox.m". "gt_mat_path_root": full path to the root directory where the .mat files, containing annotations based in bounding boxes, are located. This would correspond to the parameter "gt_mat_export_path" from "GT_bbox_GT_mat.m". As examples: '/media/abn/Data1/Databases/NVD/GT_mat' '/media/abn/Data1/Databases/NVD/GT' "gt_json_export_path": full path to the root directory annotations based in bounding boxes, will be stored in .json files. The same structure to arrange the output files as for the inputs, will be created, and this parameter can be the same path as stablished for the previous paths to export ground-truth files, or it can be another. As examples: '/media/abn/Data1/Databases/NVD/GT' '/media/abn/Data1/Databases/NVD/GT_json' 2) Run the script. 3) The output is a .json file with the ground-truth annotations based in bounding boxes of the vehicles contained in the selected sequence. The file has the name "seq#t#.json", with #t# the number of sequence. It follows the format specified in the COCO Dataset, which is compatible with Faster-RCNN.