fastfrank

Hi,
I desperatley need to edit a text log file which is 900meg big. Only problem is, it seems to take ages to load and locks the server up. Can anyone suggest a good text editor that loads directly from disc rather than trying to load it all into RAM?


Cheers
FF

zhastaph

900meg ??!!!??!?

fastfrank

yes indeedy, its a proxy server log. I need to send a report to Microsoft but i can't get into it cos its so damn big.

michael_clarkson

Can't you use split and chop the file into managable chunks, or if you now what you want to change use sed or something similiar

ozzy

I used to use Squid @ work and if you left the logging on, it would just keep growing and growing. I had to do a report and the log was well over 1Gb. I used a utility to split the log file into 50Mb chunks. There's loads of free utilities out there that'll do that.

It's the one thing I hated with Squid, so I moved to ISA. At least it creates a seperate file for each day. OK, pain in the @rse to search, but I can always combine those and use grep to search.

Stefan

fastfrank

We use Proxy Server 2, and it does the same as ISA, generating a seperate file for each day. With in-excess of 3000 users and thats only one domain of several we have, the log file hits 800meg at day without fail.

The proxy keeps crashing and MS want me to filter through the file and locate what was going on when it crashed.

Maybe I'll hunt for a splitter but i'm sure there must be something that should do the job.

cheers

stevem2k

Surely if you only want to filter the file, then just punt it onto a *nix box and grep out the entries you want, or split the file into n thousand line subfiles and use emacs/vi etc.

If it's just the last few '000 lines you are interested in (before it crashed) then tail -1000 file > file.out is trivial.

There maybe someone about who can chop it up with perl if that would be easier...


Steve

ozzy

The fact that it's a 800Mb log file is probably crashing it

If you know what you're looking for, I'd use grep to search through the file. You don't need to use Unix as there's plenty of grep utilities ported to Win32.

Stefan

fastfrank

cheers for the tips guys.

PS. Nice web site Ozzy :- )

FF

judgejules

PFE or VIM

I believe VIM is better as it will seek to the place you scroll too rather than loading the whole file in one go.

HTH

~Jules

Gedi

you need a unix box

Anyway, if you know what your looking for follow Steve's advise. There is a port of grep for windows http://www.wingrep.com

Also, again as Steve says, a perl script is the best idea for splitting the file too. I'm assuming if you had perl you can build your own, however I can throw one together.


As other have said, will emacs or vim not open something of this size?? I have never tried it as my log files are backed up daily and never get that large.
Can you get back with your outcome, I'm actually quite interested in this (yep, I need help), and I don't want to have to go to the trouble of creating a 900Mb file of random chars.