Hello,
I am not sure if my problem related to Python (I am using 2.7) or PyQt or Qt itself
I am developing log4j log file viewer. My program loads log file and parses each line to build model for QTreeView. In parallel it collects statistics about some field values. This is done using map <field_value, boolean>
Now, I have 3 test files – test1.log, test2.log, test3.log. test2.log is actually concatenation of test1.log. test3.log is also concatenation of test1.log 3 times. So set of values and thus size of maps must be the same in all 3 cases:
test1.log – ~ 50 MB and 383377 records
test2.log – ~ 100 MB and 383377 × 2 records
test3.log – ~ 150 MB and 383377 × 3 records
Now the problem:
test1 and test2 are loaded in 1-2 second and pythonw process takes ~ 35 MB of RAM
test3 loads several minutes and pythonw process takes almost 3 GB of RAM
Does anyone have a clue what could be a reason for such performance hit?
Thank you!
↧