Skip to content
Anonymous edited this page Sep 1, 2012 · 5 revisions

`[[TOC]]

Features

  • Interpolation and Binary Search
  • Compare between the two
  • Output to file
  • Intelligent date parsing - e.g. 5 minutes around a range.
  • Seamless handling of bzip2/pbzip files

SO Links

http://stackoverflow.com/questions/744256/reading-huge-file-in-python http://stackoverflow.com/questions/8369175/binary-search-over-a-huge-file-with-unknown-line-length http://stackoverflow.com/questions/4999340/python-random-access-file http://www.gossamer-threads.com/lists/python/python/812644 http://stackoverflow.com/questions/620367/python-how-to-jump-to-a-particular-line-in-a-huge-text-file

Existing Tools

http://gitorious.org/bs_grep/ http://serverfault.com/questions/101744/fast-extraction-of-a-time-range-from-syslog-logfile

Binary Search

http://wordaligned.org/articles/binary-search-revisited http://stackoverflow.com/questions/5217650/how-do-i-perform-binary-search-on-a-text-file-to-search-a-keyword-in-python

Other Links

http://www.daniweb.com/software-development/python/threads/311267/any-possible-way-to-speed-up-file-io

Seeking in Python

http://docs.python.org/library/stdtypes.html?highlight=seek#file.seek http://www.wellho.net/resources/ex.php4?item=y110/fff http://learnpythonthehardway.org/book/ex20.html http://www.java2s.com/Tutorial/Python/0240__File/Movearoundwithinthefileusingseek.htm http://www.sthurlow.com/python/lesson10/ http://stackoverflow.com/questions/4999340/python-random-access-file http://www.tutorialspoint.com/python/file_seek.htm

#C++'s Fseek http://www.cplusplus.com/reference/clibrary/cstdio/fseek/

Getting Size of File

http://stackoverflow.com/questions/6591931/getting-file-size-in-python http://docs.python.org/library/os.html#os.stat

Clone this wiki locally