Problem with recv() from website.

1) doesn't work in fact, because '\0' just stopps the string after it.

I found that it was adding at the end of buf[], so I could just add a "{" and "}" before and after, like this:

Quote:

do { { char buf[10000]; bytes = recv(googleport, buf, sizeof(buf), 0); buffer += buf; } } while (bytes > 0);

But this way I get a lot of empty chars. And it's annoying too because I don't know the size of buf, so I set it to a high number of chars, using a lot of memory. Thanks to anyone who could help.

There is no semantic difference with or without the extra set of braces.

It seems to me that your question is "how do I parse a HTTP result stream," rather than anything specific to networking. The networking will give you a chunk of data in memory, which is the HTTP reply. It may arrive in more than one call to recv(), so you need to parse the actual stream, and look for the Content-size header to know how much to read (especially with HTTP 1.1 which keeps the connection open).

One example of how to do this is found in the HTTP-GET code sample.

Depending on the http version, you may not get a Content-Length line from the server. I've been experimenting with this stuff myself lately, and the HTTP/1.1 standard appears to be "chunked encoding" where you have a hexadecimal number (after the header and the empty line) that gives you the size of the page contents. Then after the contents, it has a line with a zero on it which means there are no more chunks to transmit.

At least the website I'm using to test (my own) doesn't give a Content-Length in the header at all. If I declare HTTP/1.0 in the GET request, it says Connection: close and just closes the connection when it has sent the contents. The server is running Apache.

I'm using HTTP 1.0. But without the extra brackets, char buf[100000] doesn't get deleted after the '}' if "while" yields true. So it seems to recopy the same chars again, just adding the new one at the end (seen in debugger). That's my actual problem.

You can't guarantee that buf gets deleted like that. And if you could, that would just change the bug from copying the wrong thing to copying memory you're not allowed access to. Your problem is that you're adding buf to the string, but thinking that buf magically knows how much data you just wrote to it. It doesn't. You need to only add to buffer the exact amount of data you just read into buf. You have that information in the bytes variable.

Ah thank you ! I understand now :)

I made that:

Quote:

string buffer; int bytes; do { char buf[512]; bytes = recv(googleport, buf, sizeof(buf), 0); buffer.insert(buffer.size(), buf, bytes); } while (bytes > 0);

I added this: "buffer.insert(buffer.size(), buf, bytes);", to copy bytes of buf at the end of buffer.

I think it's fine.

Thanks guys :)

I have a stupid question. I wonder how I could avoid having to set a big char array (buf), but to still get all the data very quickly (with no data loss) ? Is there any way ?

Thank you.

Quote:

Original post by Jary I have a stupid question. I wonder how I could avoid having to set a big char array (buf), but to still get all the data very quickly (with no data loss) ? Is there any way ? Thank you.

Any time taken to resize the buffer will be completely oblitterated by the time taken to transfer the data across the network. You're talking several 10s of milliseconds to transfer the data, and a fraction of a millisecond to resize a buffer. In short, your method of adding 512 byte chunks should be fine. You might want to increase the chunk size a bit (to 4k or so) to give you slightly better performance and less memory fragmentation at the cost of more wasted space.

I would probably use an std::vector. Each time I'd try to read as many bytes as it can hold, and if I did actually receive that many, I'd increase the size of the vector so that next time, I can read more. Eventually your vector will be big enough to hold as much data as you can read in one go, hopefully making it optimally efficient as the transfer continues.

Thanks guys, this helped a lot !