Brain work
First discovery: I trained on T1w/T1w_acpc_dc, they registered T1w_acpc_dc_restore. This is fine for eval since I have both types for all images, but is a problem for fair comparison: my library is getting a huge advantage by operating on skull stripped images. In addition, this means I can't just use their atlas- will have to build my own to do a completely fair comparison.
I have sent email asking why they are not skull stripping- perhaps they have an excellent reason and I need to retrain with skulls, or perhaps they should just stick to ants and add a stripping step. Hard to say without more information.
Evalutation methodology: To get a DICE score for their current approach for a pair of images, I warp both segmentations to their atlas using the script they sent, then calculate DICE
To get a DICE score for ICON for a pair of images, I warp the segmentation from one to the other image, then calculate DICE. This will change once I have an ICON atlas built in the next few days.
Noting that this comparison is not fair in the above ways, it looks like ICON performs significantly better than their current approach. On 20 random pairs:
icon 0.7813606001925126
ants 0.7789688449216076
icon 0.8112338874457518
ants 0.7376296535317746
icon 0.7510346383760416
ants 0.7248291495038647
icon 0.7700213136841512
ants 0.7562280845695204
icon 0.7853846289592757
ants 0.7274173689692469
icon 0.7979605680416916
ants 0.7789420158351252
icon 0.8022816481887436
ants 0.7866718395036651
icon 0.7769643622031902
ants 0.7633299542410698
icon 0.7849995316673999
ants 0.7692476035458115
icon 0.8022278838905592
ants 0.7981401433813583
icon 0.7829331480423576
ants 0.7350234227770464
icon 0.7960302870982263
ants 0.7587897969101887
icon 0.7949464880607243
ants 0.7238468175069607
icon 0.8004715079602066
ants 0.7683031972517346
icon 0.7848069072434211
ants 0.7486039424706866
icon 0.775226103809265
ants 0.751596807573592
icon 0.8031604155693924
ants 0.7877281192486247
icon 1.0
ants 1.0
icon 0.7673950323965455
ants 0.7551630597562038
icon 0.7734548800820918
ants 0.7603287056802476
final
icon 0.7970946916455773
ants 0.7705394263589164Very promising! But I am not ready to share these results with colaborators yet. First I need to
convince them to switch to skull stripping, or train with skulls on
train a full resolution brain model- current results are half res while ants is using the full res images. This should hugely boost ICON's performance
compare number of folds
create an atlas and do both computations in atlas space
get raul to read my evaluation code and find where I messed up
Monai work with Basar
Collaborated with Basar to update usability of ICON with monai. Need to check in again with him to see how that is going
ICON pretrained models optimization for OAI pipeline
Fully integrated Lin's bug fix about persistent identity maps, released new pre-trained models that are ~300 mb instead of ~1.2GB. Need to put into pypi, then will integrate into oai_analysis_2
Back to Reports