![]() So we just need to call the ReadLine function until isPrefix becomes false then we can go for the next line of our file. If the line is too long and ReadLine cannot put all of its content in the buffer, It returns the filled buffer and set isPrefix to true which means we will give the next part of the line in the next call of the ReadLine function. We ignore the second parameter that we gave from the ReadLine function and that one is what we exactly need to solve our problem. Log.Fatalf( "a real error happened here: %v\n", err)Īs you probably already know, We just read the file so far, we did actually solve the problem that we had with the gigantic lines. It's not going to be a real error for us, so we have to handle it too. The other difference is the error that we will give from the ReadLine function, which can also be io.EOF. ![]() Just like the scanner, we need to call this function in a for loop but since we are at the lower level! we don't have a nice-easy boolean in return anymore to know that we can break the loop. Reader, has ReadLine function which tries to read the entire line. First we need a reader: reader := bufio.NewReader(file) generally speaking, when you hear the word lower-level you should do more for simple things but you have more access and power! ![]() It's a little bit lower level than the scanner. in this case, Scanner cannot satisfy what we need so let's take a look at ReadLine function of bufio.Reader. The next way to read such a tough file! is using another function! Bufio (buffer-io) gave us way more than a simple scanner to work with files and we have to choose one of them based on our needs and requirements. ![]() It's good but what if we need more? We can increase this number as much as we want (probably) but if our file has 5.000.000 rows and just one of them is 100Mb we need to increase our scanner to this size just for one line, or use another approach! Approach2 Now we can process jsonl files with lines up to 2Mb. Here's an example of a jsonl file.Įach line of this file represents the data of a world cup. So if we read the file line by line, we can Marshal/Unmarshal each line of it separately. What's jsonl? it's json lines in a simple term, it's a file that each line of it represents a valid json object. Today, I am going to show you how to read files in golang line-by-line.
0 Comments
Leave a Reply. |