Datasets
Models
Downloads: 1
49 lines (47 with data), 5.4 kB
÷ìY ã @ s d Z d d l m Z m Z m Z d d l Z d d l Z d d l Z d d l
Z d d l m
Z
d d l m Z d d l m Z d d l Z d d l m Z d d l m Z e d Z e d
Z d Z d Z d g Z d
Z d Z d Z d Z d Z Gd d d Z! d S)z? It reads and parses the variables, then it generate features.
é )ÚListÚTypeVarÚDictN)Úpartial)ÚCounter)ÚReadersWriters)ÚFeatureParserThread)Ú CONSTANTSÚ DataFrameÚndarrayzMohsen Mesgarpourz-Copyright 2016, https://github.com/mesgarpourÚGPLz1.1zmohsen.mesgarpour@gmail.comÚReleasec @ s e Z d Z e e e d d d Z e e e e e d d d Z e e e e e d d d Z e e e
d
d d Z d
S)Ú
FeatureParser)Úvariables_settingsÚ output_pathÚ output_tablec C s\ t j t j | _ | j j t | | _ | | _ | | _ t
| _ t | _
d S)z²Initialise the objects and constants.
:param variables_settings:
:param output_path: the output path.
:param output_table: the output table name.
N)ÚloggingÚ getLoggerr Úapp_nameÚ_FeatureParser__loggerÚdebugÚ__name__Ú"_FeatureParser__variables_settingsÚ_FeatureParser__output_pathÚ _FeatureParser__output_tabler Ú _FeatureParser__readers_writersr Ú#_FeatureParser__FeatureParserThread)Úselfr r r © r úNC:\Users\eagle\Documents\GitHub\Analytics_UoW\TCARER\Features\FeatureParser.pyÚ__init__0 s zFeatureParser.__init__)Ú
history_tableÚfeaturesÚ variablesÚ
prevalenceÚreturnc C sN | j | j d | k } x-| j D] \ } } | j j d | d d t j | d sý | d j d d j d } | j | | d | d | | | d } x t t
| D]6 }
| d d
| |
} | d d
|
f | | <qÀ Wq' t j | | d } t j
| t j d k d | } | | | d <q' W| S)
a~
:param history_table: the source table alias name (a.k.a. history table name) that features belong to
(e.g. inpatient, or outpatient).
:param features: the output features.
:param variables: the input variables.
:param prevalence: the prevalence dictionary of values for all the variables.
:return: the output features.
ÚTable_History_Namez
variable: Ú
Variable_Namez ...ÚVariable_Aggregationú Ú ú,ZVariable_Type_OriginalÚ_Nr )r Úiterrowsr ÚinfoÚpdÚisnullÚreplaceÚsplitÚ_FeatureParser__aggregateÚrangeÚlenÚnpÚ
nan_to_numÚwhereÚarray) r r! r" r# r$ r r, ÚrowÚ postfixesÚ
features_tempÚpÚ feature_namer r r ÚgenerateA s !$zFeatureParser.generate)ÚvariableÚ
variable_typer; r$ r% c C s¤ yA t j . } | j t | j j | | | | } Wd QRXWnM t k
r } z- | j j t d t
| t j WYd d } ~ Xn Xt
j | } | S)a.
:param variable: the input variable.
:param variable_type: the type of input variable.
:param postfixes: name of the aggregation functions.
:param prevalence: the prevalence dictionary of values for all the variables.
:return: the aggregated variable.
Nz - Invalid configuration(s): )ÚmpÚPoolÚmapr r Zaggregate_cellÚ
ValueErrorr Úerrorr ÚstrÚsysÚexitr6 Úasarray)r r@ rA r; r$ Úpoolr< Ú exceptionr r r Z __aggregateg s
, zFeatureParser.__aggregate)r@ Ú
variable_namer% c C s y8 t j % } | j t | j j | } Wd QRXWnM t k
r } z- | j j t d t
| t j WYd d } ~ Xn Xd d | D } t
| j } | j j | j | j | d j d d | D g d d d d
d d | D } | S) zµ
:param variable: the input variable.
:param variable_name: the name of the input variable.
:return: the prevalence of values for all the variables.
Nz - Invalid configuration(s): c S s# g | ] } | D] } | q q Sr r )Ú.0Úsub1Úsub2r r r ú
<listcomp> s z,FeatureParser.prevalence.<locals>.<listcomp>z; c S s2 g | ]( } t | d d t | d q S)r ú:é )rG )rN r= r r r rQ s ÚappendTÚextÚtxtc S s g | ] } | d q S)r r )rN r= r r r rQ s )rB rC rD r r Zprevalence_cellrE r rF r rG rH rI r Ú most_commonr Ú save_textr r Újoin)r r@ rM rK Zprevalence_temprL r$ r r r r$ s # "
zFeatureParser.prevalenceN) r Ú
__module__Ú __qualname__ÚPandasDataFramerG r r r? Ú NumpyNdarrayr3 r r$ r r r r r . s #r )"Ú__doc__Útypingr r r rH Úpandasr/ Únumpyr6 ÚmultiprocessingrB Ú functoolsr Ú collectionsr Ú ReadersWriters.ReadersWritersr r Z Features.FeatureParserThreadr ÚConfigs.CONSTANTSr r\ r] Ú
__author__Ú
__copyright__Ú __credits__Ú __license__Ú __version__Ú__maintainer__Ú __email__Ú
__status__r r r r r Ú<module> s,