How to read a specific line from a very large file in Java

So the other day I had to write some Java code to read specific lines of text from very large files full of ASCII text records (of a fixed length). The records in the files are generally defined by the the length of a record (no carriage returns, fixed length, i.e. each “record” is 290 bytes in length). Either way given a large file, I had the need to “jump to record/line 12400 and read until 14500). Great….since these are very large files and I want to go to / read a particular line of text, I don’t really want to open up the entire thing and sequentially do a reader.readLine() or reader.read(buffer, 0, sizeOfEachRecordInBytes) while keeping count until I get to the ones I am interested in. Instead you can do it as follows by leveraging RandomAccessFile.seek() or BufferedReader.skip() since you know the lengths:

int sizeofrecordinbytes = 290;
 // for this example this is 1 based, not zero based 
int recordIWantToStartAt = 12400;
int totalRecordsIWant = 1000;

File myfile = new File("someGiantFile.txt");


// where to seek to
long seekToByte =  (recordIWantToStartAt == 1 ? 0 : ((recordIWantToStartAt-1) * sizeofrecordinbytes));

// byte the reader will jump to once we know where to go
long startAtByte = 0;

// seek to that position using a RandomAccessFile
try {
        // NOTE since we are using fixed length records, you could actually skip this 
        // and just use our seekToByte as the value for the BufferedReader.skip() call below

	RandomAccessFile rand = new RandomAccessFile(myfile,"r");
	rand.seek(seekToByte);
	startAtByte = rand.getFilePointer();
	rand.close();
	
} catch(IOException e) {
	// do something
}

// Do it using the BufferedReader 
BufferedReader reader = null;
try {
	// lets fire up a buffered reader and skip right to that spot.
	reader = new BufferedReader(new FileReader(myfile));
	reader.skip(startAtByte);
	
	String line;
	long totalRead = 0;
	char[] buffer = new char[sizeofrecordinbytes];
	while(totalRead < totalRecordsIWant && (-1 != reader.read(buffer, 0, sizeofrecordinbytes))) {
		System.out.println(new String(buffer));
		totalRead++;
	}
} catch(Exception e) {
	// handle this
	
} finally {
	if (reader != null) {
		try {reader.close();} catch(Exception ignore) {}
	}
}

Hope that gets you started in figuring out how to seek to / jump to specific lines or records within a text file using Java!

Advertisements

4 comments

  1. Rajat

    I am faced with a similar problem, but the only thing which has kept me thinking is: I dont know the line number I need to jump to, Please give a shout if you got ideas?

    Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s