Commits · 821243cdc9b3d6d982f38099ea45311ff030bff0 · Jurica Seva / clef18

Jun 20, 2018

Fix seq2seq attention model · 821243cd
Mario Sänger authored 6 years ago

821243cd

Added gold standard data.seq2seq with attention. Training works. Inference models have an error: · 5b5b290b

Jurica Seva authored 6 years ago

Traceback (most recent call last):
File "seq2seq_attention.py", line 163, in <module>
outputs=[decoder_out] + decoder_states )
File "/home/sevajuri/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/sevajuri/anaconda3/lib/python3.6/site-packages/keras/engine/topology.py", line 1811, in __init__
str(layers_with_complete_input))
RuntimeError: Graph disconnected: cannot obtain value for tensor Tensor("input_1_1:0", shape=(?, 27), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: ['input_2']

5b5b290b

Jun 15, 2018
- Added gold standard data. · 1fb73198
  Jurica Seva authored 6 years ago
  
  1fb73198
May 31, 2018
- Final version. All info included in to the writing. Fixed typos. Added 4.4. Formated tables et al. · 77b166c8
  Jurica Seva authored 6 years ago
  
  77b166c8
May 25, 2018
- Updated with results on the train/val dataset. · 87a185ed
  Jurica Seva authored 6 years ago
  
  87a185ed
May 24, 2018
- Started writeup. · 935f6903
  Jurica Seva authored 6 years ago
  
  935f6903
May 23, 2018
- Final results. · 7b798988
  Jurica Seva authored 6 years ago
  
  7b798988
May 17, 2018
- Final results. · c40f91fb
  Jurica Seva authored 6 years ago
  
  c40f91fb
May 14, 2018
- Generating results. · 631681b8
  Jurica Seva authored 6 years ago
  
  631681b8
- Generating results. · 3629afdb
  Jurica Seva authored 6 years ago
  
  3629afdb
May 12, 2018
- Extend method description · a00cc920
  Mario Sänger authored 6 years ago
  
  a00cc920
May 11, 2018
- Generating results. · 48634b43
  Jurica Seva authored 6 years ago
  
  48634b43
May 10, 2018
- Tested duplicated ICD10 codes. Same results. · 8d1e8f63
  Jurica Seva authored 6 years ago
  
  8d1e8f63
May 09, 2018
- fit_generator implemented. Slow as hell. · e6a9c021
  Jurica Seva authored 6 years ago
  
  e6a9c021
May 08, 2018
- Add support for multi code lines · 027c22c4
  Mario Sänger authored 6 years ago
  
  027c22c4
May 07, 2018
- fit_generator implemented. Slow as hell. · 46df106c
  Jurica Seva authored 6 years ago
  
  46df106c
May 05, 2018

Started working on a pipeline using all data. Issues with memory (RAM) while... · 535a7a63

Jurica Seva authored 6 years ago

Started working on a pipeline using all data. Issues with memory (RAM) while generating one-hot label encodings. Solutions: use data generators to train via fit_generator. Examples:
https://github.com/keras-team/keras/issues/1627
https://www.kaggle.com/ezietsman/simple-keras-model-with-data-generator
https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly.html

535a7a63

Final model with balanced s2s training data and ICD10 without additional data... · b536c5a8
Jurica Seva authored 6 years ago
```
Final model with balanced s2s training data and ICD10 without additional data points from CC/CB combination.
```
b536c5a8

May 04, 2018

Added ICD 10 classification with attention (best results), attention with... · da2a097e

Jurica Seva authored 6 years ago

Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).

da2a097e

May 03, 2018

Added ICD 10 classification with attention (best results), attention with... · b202a46b

Jurica Seva authored 6 years ago

Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).

b202a46b

Added ICD 10 classification with attention (best results), attention with... · e6a59bea

Jurica Seva authored 6 years ago

Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).

e6a59bea

Added ICD 10 classification with attention (best results), attention with... · 39db3e78

Jurica Seva authored 6 years ago

Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).

39db3e78

Apr 30, 2018

Added ICD 10 classification with attention (best results), attention with... · 0eb9f0ec

Jurica Seva authored 6 years ago

Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).

0eb9f0ec

Apr 28, 2018

Merge remote-tracking branch 'origin/master' · 518e8e19

Jurica Seva authored 6 years ago

# Conflicts:
#	code/data/train/FR/training/aligned/PavillonLaurent.pdf
#	code/data/train/FR/training/aligned/README.txt
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2006-2012full.csv
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2013.csv
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2013full.csv
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2014.csv
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2014_full.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2006-2010.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2011.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2012.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2013.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2014.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2015.csv
#	code/data/train/FR/training/aligned/eval/clefehealth2017Task1_eval.pl
#	code/data/train/FR/training/raw/README.txt
#	code/data/train/FR/training/raw/corpus/CausesBrutes_FR_2006-2012.csv
#	code/data/train/FR/training/raw/corpus/CausesBrutes_FR_2013.csv
#	code/data/train/FR/training/raw/corpus/CausesBrutes_FR_2014.csv
#	code/data/train/FR/training/raw/corpus/CausesCalculees_FR_2006-2012.csv
#	code/data/train/FR/training/raw/corpus/CausesCalculees_FR_2013.csv
#	code/data/train/FR/training/raw/corpus/CausesCalculees_FR_2014.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2006-2012_full.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2013.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2013_full.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2014.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2014_full.csv
#	code/data/train/FR/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code/data/train/HU/training/aligned/README.txt
#	code/data/train/HU/training/raw/README.txt
#	code/data/train/HU/training/raw/certificate_HUN16_EN.xls
#	code/data/train/HU/training/raw/certificate_HUN16_HU.xlsx
#	code/data/train/HU/training/raw/corpus/CausesBrutes_HU_1.csv
#	code/data/train/HU/training/raw/corpus/CausesCalculees_HU_1.csv
#	code/data/train/HU/training/raw/corpus/Ident_HU_1.csv
#	code/data/train/HU/training/raw/dictionaries/Hungarian_dictionary_UTF8.csv
#	code/data/train/HU/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code/data/train/IT/training/aligned/README.txt
#	code/data/train/IT/training/raw/Certificate_Italian.pdf
#	code/data/train/IT/training/raw/README.txt
#	code/data/train/IT/training/raw/corpus/CausesBrutes_IT_1.csv
#	code/data/train/IT/training/raw/corpus/CausesCalculees_IT_1.csv
#	code/data/train/IT/training/raw/corpus/Ident_IT_1.csv
#	code/data/train/IT/training/raw/dictionaries/dictionary_IT.csv
#	code/data/train/IT/training/raw/eval/#clefehealthTask12017_plainCertifeval.pl#
#	code/data/train/IT/training/raw/eval/.#clefehealthTask12017_plainCertifeval.pl
#	code/data/train/IT/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code/data/train/_zip/CLEFeHealth2018Task1_HU_training.zip
#	code/data/train/_zip/CLEFeHealth2018Task1_IT_training.zip
#	code/data/train/_zip/training_CLEFeHealth2018_FR.zip
#	code_jurica/data/train/FR/training/aligned/PavillonLaurent.pdf
#	code_jurica/data/train/FR/training/aligned/README.txt
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2006-2012full.csv
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2013.csv
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2013full.csv
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2014.csv
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2014_full.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2006-2010.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2011.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2012.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2013.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2014.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2015.csv
#	code_jurica/data/train/FR/training/aligned/eval/clefehealth2017Task1_eval.pl
#	code_jurica/data/train/FR/training/raw/README.txt
#	code_jurica/data/train/FR/training/raw/corpus/CausesBrutes_FR_2006-2012.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesBrutes_FR_2013.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesBrutes_FR_2014.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesCalculees_FR_2006-2012.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesCalculees_FR_2013.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesCalculees_FR_2014.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2006-2012_full.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2013.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2013_full.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2014.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2014_full.csv
#	code_jurica/data/train/FR/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_jurica/data/train/HU/training/aligned/README.txt
#	code_jurica/data/train/HU/training/raw/README.txt
#	code_jurica/data/train/HU/training/raw/certificate_HUN16_EN.xls
#	code_jurica/data/train/HU/training/raw/certificate_HUN16_HU.xlsx
#	code_jurica/data/train/HU/training/raw/corpus/CausesBrutes_HU_1.csv
#	code_jurica/data/train/HU/training/raw/corpus/CausesCalculees_HU_1.csv
#	code_jurica/data/train/HU/training/raw/corpus/Ident_HU_1.csv
#	code_jurica/data/train/HU/training/raw/dictionaries/Hungarian_dictionary_UTF8.csv
#	code_jurica/data/train/HU/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_jurica/data/train/IT/training/aligned/README.txt
#	code_jurica/data/train/IT/training/raw/Certificate_Italian.pdf
#	code_jurica/data/train/IT/training/raw/README.txt
#	code_jurica/data/train/IT/training/raw/corpus/CausesBrutes_IT_1.csv
#	code_jurica/data/train/IT/training/raw/corpus/CausesCalculees_IT_1.csv
#	code_jurica/data/train/IT/training/raw/corpus/Ident_IT_1.csv
#	code_jurica/data/train/IT/training/raw/dictionaries/dictionary_IT.csv
#	code_jurica/data/train/IT/training/raw/eval/#clefehealthTask12017_plainCertifeval.pl#
#	code_jurica/data/train/IT/training/raw/eval/.#clefehealthTask12017_plainCertifeval.pl
#	code_jurica/data/train/IT/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_jurica/data/train/_zip/CLEFeHealth2018Task1_HU_training.zip
#	code_jurica/data/train/_zip/CLEFeHealth2018Task1_IT_training.zip
#	code_jurica/data/train/_zip/training_CLEFeHealth2018_FR.zip
#	code_mario/data/train/FR/training/aligned/PavillonLaurent.pdf
#	code_mario/data/train/FR/training/aligned/README.txt
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2006-2012full.csv
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2013.csv
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2013full.csv
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2014.csv
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2014_full.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2006-2010.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2011.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2012.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2013.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2014.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2015.csv
#	code_mario/data/train/FR/training/aligned/eval/clefehealth2017Task1_eval.pl
#	code_mario/data/train/FR/training/raw/README.txt
#	code_mario/data/train/FR/training/raw/corpus/CausesBrutes_FR_2006-2012.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesBrutes_FR_2013.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesBrutes_FR_2014.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesCalculees_FR_2006-2012.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesCalculees_FR_2013.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesCalculees_FR_2014.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2006-2012_full.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2013.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2013_full.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2014.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2014_full.csv
#	code_mario/data/train/FR/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_mario/data/train/HU/training/aligned/README.txt
#	code_mario/data/train/HU/training/raw/README.txt
#	code_mario/data/train/HU/training/raw/certificate_HUN16_EN.xls
#	code_mario/data/train/HU/training/raw/certificate_HUN16_HU.xlsx
#	code_mario/data/train/HU/training/raw/corpus/CausesBrutes_HU_1.csv
#	code_mario/data/train/HU/training/raw/corpus/CausesCalculees_HU_1.csv
#	code_mario/data/train/HU/training/raw/corpus/Ident_HU_1.csv
#	code_mario/data/train/HU/training/raw/dictionaries/Hungarian_dictionary_UTF8.csv
#	code_mario/data/train/HU/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_mario/data/train/IT/training/aligned/README.txt
#	code_mario/data/train/IT/training/raw/Certificate_Italian.pdf
#	code_mario/data/train/IT/training/raw/README.txt
#	code_mario/data/train/IT/training/raw/corpus/CausesBrutes_IT_1.csv
#	code_mario/data/train/IT/training/raw/corpus/CausesCalculees_IT_1.csv
#	code_mario/data/train/IT/training/raw/corpus/Ident_IT_1.csv
#	code_mario/data/train/IT/training/raw/dictionaries/dictionary_IT.csv
#	code_mario/data/train/IT/training/raw/eval/#clefehealthTask12017_plainCertifeval.pl#
#	code_mario/data/train/IT/training/raw/eval/.#clefehealthTask12017_plainCertifeval.pl
#	code_mario/data/train/IT/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_mario/data/train/_zip/CLEFeHealth2018Task1_HU_training.zip
#	code_mario/data/train/_zip/CLEFeHealth2018Task1_IT_training.zip
#	code_mario/data/train/_zip/training_CLEFeHealth2018_FR.zip

518e8e19

Apr 27, 2018
- Finished pipeline: s2s -> Text Classification -> ICD10 · fb29aef3
  Jurica Seva authored 6 years ago
  
  fb29aef3
- Add second code folder · 5487e75f
  Mario Sänger authored 6 years ago
  
  5487e75f