Commits · 23c9c793919f8d377927cce24488f192ec239d65 · Jurica Seva / clef18

May 07, 2018
- Fix minor naming issue in embedding model 2 · 23c9c793
  Mario Sänger authored 6 years ago
  
  23c9c793
- fit_generator implemented. Slow as hell. · 46df106c
  Jurica Seva authored 6 years ago
  
  46df106c
May 05, 2018

Started working on a pipeline using all data. Issues with memory (RAM) while... · 535a7a63

Jurica Seva authored 6 years ago

Started working on a pipeline using all data. Issues with memory (RAM) while generating one-hot label encodings. Solutions: use data generators to train via fit_generator. Examples:
https://github.com/keras-team/keras/issues/1627
https://www.kaggle.com/ezietsman/simple-keras-model-with-data-generator
https://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly.html

535a7a63

Merge remote-tracking branch 'origin/master' · 86e59d9c
Jurica Seva authored 6 years ago

86e59d9c
Final model with balanced s2s training data and ICD10 without additional data... · b536c5a8
Jurica Seva authored 6 years ago
```
Final model with balanced s2s training data and ICD10 without additional data points from CC/CB combination.
```
b536c5a8
Add initial version of embedding model 2 · 7ee72cb2
Mario Sänger authored 6 years ago

7ee72cb2

May 04, 2018
- Save program arguments to file · 869d45a8
  Mario Sänger authored 6 years ago
  
  869d45a8
- Extend debugging options · 4122be21
  Mario Sänger authored 6 years ago
  
  4122be21
- Rename execution script · 64ba64a7
  Mario Sänger authored 6 years ago
  
  64ba64a7
- Add random seed initialization · a859dea8
  Mario Sänger authored 6 years ago
  
  a859dea8
- Minor updates of classifiers · d6503a3f
  Mario Sänger authored 6 years ago
  
  d6503a3f
- Merge remote-tracking branch 'origin/master' · 7f639693
  Jurica Seva authored 6 years ago
  
  7f639693
- Added ICD 10 classification with attention (best results), attention with... · da2a097e
  Jurica Seva authored 6 years ago
```
Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).
```
  da2a097e
May 03, 2018
- Added extended negative sampling strategy + fixed test-train splitting · a19ac541
  Mario Sänger authored 6 years ago
  
  a19ac541
- Add selection of language · 28e6a2ed
  Mario Sänger authored 6 years ago
  
  28e6a2ed
- Add ICD10 code masking (by code frequency) · 068ad3fe
  Mario Sänger authored 6 years ago
  
  068ad3fe
- Fix stratified column · 032dab16
  Mario Sänger authored 6 years ago
  
  032dab16
- Added ICD 10 classification with attention (best results), attention with... · b202a46b
  Jurica Seva authored 6 years ago
```
Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).
```
  b202a46b
- Added ICD 10 classification with attention (best results), attention with... · e6a59bea
  Jurica Seva authored 6 years ago
```
Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).
```
  e6a59bea
- Merge remote-tracking branch 'origin/master' · d0c34b11
  Jurica Seva authored 6 years ago
  
  d0c34b11
- Added ICD 10 classification with attention (best results), attention with... · 39db3e78
  Jurica Seva authored 6 years ago
```
Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).
```
  39db3e78
May 01, 2018
- Extend .gitignore to exclude training data · 9807cad9
  Mario Sänger authored 6 years ago
  
  9807cad9
- Add possibility to select target column · 5b7d4b73
  Mario Sänger authored 6 years ago
  
  5b7d4b73
Apr 30, 2018

Added ICD 10 classification with attention (best results), attention with... · 0eb9f0ec

Jurica Seva authored 6 years ago

Added ICD 10 classification with attention (best results), attention with context (worst results) and char level classification (running atm).

0eb9f0ec

Apr 28, 2018

Merge remote-tracking branch 'origin/master' · 518e8e19

Jurica Seva authored 6 years ago

# Conflicts:
#	code/data/train/FR/training/aligned/PavillonLaurent.pdf
#	code/data/train/FR/training/aligned/README.txt
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2006-2012full.csv
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2013.csv
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2013full.csv
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2014.csv
#	code/data/train/FR/training/aligned/corpus/AlignedCauses_2014_full.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2006-2010.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2011.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2012.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2013.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2014.csv
#	code/data/train/FR/training/aligned/dictionaries/Dictionnaire2015.csv
#	code/data/train/FR/training/aligned/eval/clefehealth2017Task1_eval.pl
#	code/data/train/FR/training/raw/README.txt
#	code/data/train/FR/training/raw/corpus/CausesBrutes_FR_2006-2012.csv
#	code/data/train/FR/training/raw/corpus/CausesBrutes_FR_2013.csv
#	code/data/train/FR/training/raw/corpus/CausesBrutes_FR_2014.csv
#	code/data/train/FR/training/raw/corpus/CausesCalculees_FR_2006-2012.csv
#	code/data/train/FR/training/raw/corpus/CausesCalculees_FR_2013.csv
#	code/data/train/FR/training/raw/corpus/CausesCalculees_FR_2014.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2006-2012_full.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2013.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2013_full.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2014.csv
#	code/data/train/FR/training/raw/corpus/Ident_FR_2014_full.csv
#	code/data/train/FR/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code/data/train/HU/training/aligned/README.txt
#	code/data/train/HU/training/raw/README.txt
#	code/data/train/HU/training/raw/certificate_HUN16_EN.xls
#	code/data/train/HU/training/raw/certificate_HUN16_HU.xlsx
#	code/data/train/HU/training/raw/corpus/CausesBrutes_HU_1.csv
#	code/data/train/HU/training/raw/corpus/CausesCalculees_HU_1.csv
#	code/data/train/HU/training/raw/corpus/Ident_HU_1.csv
#	code/data/train/HU/training/raw/dictionaries/Hungarian_dictionary_UTF8.csv
#	code/data/train/HU/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code/data/train/IT/training/aligned/README.txt
#	code/data/train/IT/training/raw/Certificate_Italian.pdf
#	code/data/train/IT/training/raw/README.txt
#	code/data/train/IT/training/raw/corpus/CausesBrutes_IT_1.csv
#	code/data/train/IT/training/raw/corpus/CausesCalculees_IT_1.csv
#	code/data/train/IT/training/raw/corpus/Ident_IT_1.csv
#	code/data/train/IT/training/raw/dictionaries/dictionary_IT.csv
#	code/data/train/IT/training/raw/eval/#clefehealthTask12017_plainCertifeval.pl#
#	code/data/train/IT/training/raw/eval/.#clefehealthTask12017_plainCertifeval.pl
#	code/data/train/IT/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code/data/train/_zip/CLEFeHealth2018Task1_HU_training.zip
#	code/data/train/_zip/CLEFeHealth2018Task1_IT_training.zip
#	code/data/train/_zip/training_CLEFeHealth2018_FR.zip
#	code_jurica/data/train/FR/training/aligned/PavillonLaurent.pdf
#	code_jurica/data/train/FR/training/aligned/README.txt
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2006-2012full.csv
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2013.csv
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2013full.csv
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2014.csv
#	code_jurica/data/train/FR/training/aligned/corpus/AlignedCauses_2014_full.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2006-2010.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2011.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2012.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2013.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2014.csv
#	code_jurica/data/train/FR/training/aligned/dictionaries/Dictionnaire2015.csv
#	code_jurica/data/train/FR/training/aligned/eval/clefehealth2017Task1_eval.pl
#	code_jurica/data/train/FR/training/raw/README.txt
#	code_jurica/data/train/FR/training/raw/corpus/CausesBrutes_FR_2006-2012.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesBrutes_FR_2013.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesBrutes_FR_2014.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesCalculees_FR_2006-2012.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesCalculees_FR_2013.csv
#	code_jurica/data/train/FR/training/raw/corpus/CausesCalculees_FR_2014.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2006-2012_full.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2013.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2013_full.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2014.csv
#	code_jurica/data/train/FR/training/raw/corpus/Ident_FR_2014_full.csv
#	code_jurica/data/train/FR/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_jurica/data/train/HU/training/aligned/README.txt
#	code_jurica/data/train/HU/training/raw/README.txt
#	code_jurica/data/train/HU/training/raw/certificate_HUN16_EN.xls
#	code_jurica/data/train/HU/training/raw/certificate_HUN16_HU.xlsx
#	code_jurica/data/train/HU/training/raw/corpus/CausesBrutes_HU_1.csv
#	code_jurica/data/train/HU/training/raw/corpus/CausesCalculees_HU_1.csv
#	code_jurica/data/train/HU/training/raw/corpus/Ident_HU_1.csv
#	code_jurica/data/train/HU/training/raw/dictionaries/Hungarian_dictionary_UTF8.csv
#	code_jurica/data/train/HU/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_jurica/data/train/IT/training/aligned/README.txt
#	code_jurica/data/train/IT/training/raw/Certificate_Italian.pdf
#	code_jurica/data/train/IT/training/raw/README.txt
#	code_jurica/data/train/IT/training/raw/corpus/CausesBrutes_IT_1.csv
#	code_jurica/data/train/IT/training/raw/corpus/CausesCalculees_IT_1.csv
#	code_jurica/data/train/IT/training/raw/corpus/Ident_IT_1.csv
#	code_jurica/data/train/IT/training/raw/dictionaries/dictionary_IT.csv
#	code_jurica/data/train/IT/training/raw/eval/#clefehealthTask12017_plainCertifeval.pl#
#	code_jurica/data/train/IT/training/raw/eval/.#clefehealthTask12017_plainCertifeval.pl
#	code_jurica/data/train/IT/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_jurica/data/train/_zip/CLEFeHealth2018Task1_HU_training.zip
#	code_jurica/data/train/_zip/CLEFeHealth2018Task1_IT_training.zip
#	code_jurica/data/train/_zip/training_CLEFeHealth2018_FR.zip
#	code_mario/data/train/FR/training/aligned/PavillonLaurent.pdf
#	code_mario/data/train/FR/training/aligned/README.txt
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2006-2012full.csv
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2013.csv
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2013full.csv
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2014.csv
#	code_mario/data/train/FR/training/aligned/corpus/AlignedCauses_2014_full.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2006-2010.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2011.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2012.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2013.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2014.csv
#	code_mario/data/train/FR/training/aligned/dictionaries/Dictionnaire2015.csv
#	code_mario/data/train/FR/training/aligned/eval/clefehealth2017Task1_eval.pl
#	code_mario/data/train/FR/training/raw/README.txt
#	code_mario/data/train/FR/training/raw/corpus/CausesBrutes_FR_2006-2012.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesBrutes_FR_2013.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesBrutes_FR_2014.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesCalculees_FR_2006-2012.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesCalculees_FR_2013.csv
#	code_mario/data/train/FR/training/raw/corpus/CausesCalculees_FR_2014.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2006-2012_full.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2013.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2013_full.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2014.csv
#	code_mario/data/train/FR/training/raw/corpus/Ident_FR_2014_full.csv
#	code_mario/data/train/FR/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_mario/data/train/HU/training/aligned/README.txt
#	code_mario/data/train/HU/training/raw/README.txt
#	code_mario/data/train/HU/training/raw/certificate_HUN16_EN.xls
#	code_mario/data/train/HU/training/raw/certificate_HUN16_HU.xlsx
#	code_mario/data/train/HU/training/raw/corpus/CausesBrutes_HU_1.csv
#	code_mario/data/train/HU/training/raw/corpus/CausesCalculees_HU_1.csv
#	code_mario/data/train/HU/training/raw/corpus/Ident_HU_1.csv
#	code_mario/data/train/HU/training/raw/dictionaries/Hungarian_dictionary_UTF8.csv
#	code_mario/data/train/HU/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_mario/data/train/IT/training/aligned/README.txt
#	code_mario/data/train/IT/training/raw/Certificate_Italian.pdf
#	code_mario/data/train/IT/training/raw/README.txt
#	code_mario/data/train/IT/training/raw/corpus/CausesBrutes_IT_1.csv
#	code_mario/data/train/IT/training/raw/corpus/CausesCalculees_IT_1.csv
#	code_mario/data/train/IT/training/raw/corpus/Ident_IT_1.csv
#	code_mario/data/train/IT/training/raw/dictionaries/dictionary_IT.csv
#	code_mario/data/train/IT/training/raw/eval/#clefehealthTask12017_plainCertifeval.pl#
#	code_mario/data/train/IT/training/raw/eval/.#clefehealthTask12017_plainCertifeval.pl
#	code_mario/data/train/IT/training/raw/eval/clefehealthTask12017_plainCertifeval.pl
#	code_mario/data/train/_zip/CLEFeHealth2018Task1_HU_training.zip
#	code_mario/data/train/_zip/CLEFeHealth2018Task1_IT_training.zip
#	code_mario/data/train/_zip/training_CLEFeHealth2018_FR.zip

518e8e19

Apr 27, 2018
- Trying tu push · 0c5daf30
  Jurica Seva authored 6 years ago
  
  0c5daf30
- Finished pipeline: s2s -> Text Classification -> ICD10 · fb29aef3
  Jurica Seva authored 6 years ago
  
  fb29aef3
- Add second code folder · 5487e75f
  Mario Sänger authored 6 years ago
  
  5487e75f
- Rename code folder · cad3d02a
  Mario Sänger authored 6 years ago
  
  cad3d02a
- Adapt .gitignore file · e960d5c6
  Mario Sänger authored 6 years ago
  
  e960d5c6
- Add more configuration options · 5203106e
  Mario Sänger authored 6 years ago
  
  5203106e
Apr 24, 2018
- Add best model usage · 0976bab7
  Mario Sänger authored 6 years ago
  
  0976bab7
- Minor changes to classification · 66ddf4d0
  Mario Sänger authored 6 years ago
  
  66ddf4d0
Apr 23, 2018
- Implement embedding layer · 1a77b90f
  Mario Sänger authored 6 years ago
  
  1a77b90f
Apr 19, 2018
- Add filtering of single code lines · 3a18f269
  Mario Sänger authored 6 years ago
  
  3a18f269
- Add ICD10 chapter classification · dd41b96c
  Mario Sänger authored 6 years ago
  
  dd41b96c
Apr 17, 2018
- Add initial version of KNN classifier training · 7c75900a
  Mario Sänger authored 6 years ago
  
  7c75900a
Mar 23, 2018

Initial version of representation learning model for eCLEF18-Task1. The model... · 232704f6

Mario Sänger authored 7 years ago

Initial version of representation learning model for eCLEF18-Task1. The model is able to distinguish between correct line-ICD10-code pairs (originating from the training data) and randomly sampled corrupted pairs.

232704f6

Mar 14, 2018
- Initial commit · c2c58df2
  Jurica Seva authored 7 years ago
  
  c2c58df2
- Initial commit · 9c999b1b
  Jurica Seva authored 7 years ago
  
  9c999b1b