Skip to main content
GameDev.net gamedev.net
🔒 Locked

Exception in thread "Thread-4" java.lang.OutOfMemoryError: Java heap space

Started by Jarwulf Nov 8, 2012 at 10:57 PM 1 replies 3.4k views
Original Post
Jarwulf
Jarwulf
Hi, I'm getting a crash in Java. Basically a txt file circa 150mb is being processed


fileHandler.shrinkFile();


goes into




public void shrinkFile() throws FileNotFoundException, IOException, FileFormatException{
BufferedWriter outputTempFileWriter = null;
BufferedReader inputFileReader = null;


for(int i=0;i<choseFilesVector.size();i++){
String inputFile = choseFilesVector.get(i).toString();
inputFileReader = new BufferedReader(new FileReader( inputFile ), readBufferSize);

txtArea.append(" Processing "+inputFile+"\n");
txtArea.setCaretPosition(txtArea.getText().length());

//decide format
String strLine = null;
String format = null;
do{

//CRASH HAPPENS HERE
strLine = inputFileReader.readLine();
//CRASH HAPPENS HERE

if(strLine!=null && (strLine.split("\t").length>=4)){
String [] info = strLine.split("\t");
if((info[0].trim().toLowerCase().matches("chr(x|y|[0-9]+)"))&&(info[1].trim().matches("[0-9]+"))&&(info[2].trim().matches("[0-9]+"))){
//data of tagAlign
format = "tagAlign";


It gets into the above loop and then simply crashes on line

strLine = inputFileReader.readLine();


the first time through giving me the error

Exception in thread "Thread-4" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)


etc etc. Maybe the file I'm processing is too big but the program has happened gigantic files before the only difference this time is that the files being processed were autoconverted from a different format.
Bruno Sofiato
Bruno Sofiato
Actually its not because the file size, but I bet this new file has a gigantic line on it.

The readLine() method will try and allocate the full line as a string in memory. If the line of text gets to big, you'll eventually run out of heap.

There's two courses of action that you can take :

  1. Raise the amount of memory avaliable to the JVM (It'll work, but eventually you may run on OutOfMemory erros again)
  2. Refactor the code without relying on the readLine() method. (Instead of reading the full string and spliting it on memory, you should create a FSM and parse the file on a more "streaming" fashion)
Jarwulf
Jarwulf
Thanks, I added a newline when file generating and it seems to have worked.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.