Frequently asked questions:¶
1. The challenge description document says "task 1 is aimed at segmentation of [...] laparoscopy video images and preoperative 3D model". The given labels for the 3D model are view-dependent, however. Does this mean we are expected to produce a 3D segmentation _per image_? It seems like this would require a previous registration already (or at least parsing of the 2D images and the 3D model at the same time). Are we misinterpreting something here, or is this really the task?
=> For 2D Segmentation: You are required to develop a ML approach to determine the 2D landmark contours.
For 3D contour segmentation: Yes, the segmentation is view dependent to that of provided 2D images. However, there will be no registration parameters provided to the teams. The teams are required to develop a ML approach that will take the 2D image or the segmented landmark contours that will correspond to 3D landmarks in that view of the 3D model. We will apply a margin of tolerance to compensate for offset in the detected landmarks compared to the ground truth.
2. Do we understand correctly that we are to perform a registration between the 3D object and each of the 2D images individually?
=> Yes, this is correct. To be clear, the idea here is to use the detected 3D landmarks and the 2D contours to perform registration.
Should we consider all images of a patient together and produce one registration for them (along with the translation between the 2D images)?
=> No, we expect the participants to compute the transformation for each 2D image corresponding to the 3D contour points.
3. To calculate the reprojection error, we would require the camera parameters, but have not found them anywhere. Will they be provided?
=> Yes, we will provide this. A link is provided here for patient dataset: https://drive.google.com/file/d/1ind9ZkyutqV2_I1bRxxQGAPpU-7__hxh/view?usp=sharing
4. Where are the .png images mentioned in the challenge description? Did we only download a part of the data?
=> The provided images are currently in ‘.jpg’. The formatting could be either ‘.png’ or ‘.jpg’.
5. The description mentioned that "Participants will be allowed to choose their validation set as per their choice and the details on data will be provided to them." Could this be elaborated on? What does this mean?
=> This means that you can distribute the training and validation data as you want.
6. When will the code with metrics be supplied? When we checked, the github repository did not seem to be done yet? Seeing this code might help answer some of the questions above…
=> We are currently working on this and we hope to release and inform all participants via an email once ready.
7. Part of our methods that we are currently developing encompasses generating data synthetically to then train on. We assume that this kind of data (synthetic, and created for this task specifically) may be used, despite not being public?
=> You can use publicly available datasets but we do not allow in-house non-public datasets. You could also develop your dataset but you will have to make it public to make the method reproducible.
8. Registration transformation is rigid. And the expected matrix is 12 degrees of freedom.
=> The transformation matrix should be computed per image.
9. In the testing phase, the contours of the 3 anatomies in the CT-scan ( mesh ) will not be provided.
=> The idea is to compute the 2D and 3D contours automatically (task-I) that can then be used for task 2 registration.
10. Are we supposed to submit results of the segmentation task ?
=> Yes, you cannot participate in task 2 if you do not submit task 1 results. Task 2 depends on task 1 landmark computations.
11. Should we submit the paper ( explaining the method ) before or after 17th of August? If the results were not in the top 5, will it be published in the proceedings?
=> We will be taking the input from top 5 teams that will be integrated in the intended joint-journal paper. You will have to submit a summary of your method, preliminary results on your validation set and details on any public dataset that you have used in your experiments. The summary report can be between 2 - 4 pages long.
12. In order to evaluate my work, a registration matrix as ground truth is required. A matrix could be established from the given set of points in both modalities, however it seems more points are required from the mesh for a better correspondence.
=> For the patient data, you can compute a reprojection error. However, for the phantom data we will provide you with both intrinsic and extrinsic parameters that will serve as ground truth registration matrix.
13. In order to interpolate more points is it ok if we convert the modality of the given mesh into another?
=> Do you mean modifying the mesh? Yes, this is ok for training, however, you should not do it for test data as we will be evaluating on the original model.
14. What is the initial alignment of the video frame and the CT mesh when providing a transformation matrix? In other terms, what is the frame of reference for the video frame ?
=> The provided individual frames are reference to the set of 2D and 3D landmark contours. There will be no initial alignment provided. Please see (1) for details.
15. Will other metrics be provided for validation?
=> We will release metrics as soon as possible. However, participants at this stage can work with F1-score, precision and recall for 2D image points and MSD or Hausdorff Distance for 3D contours. For registration, we advise to use re-projection error for patient data.