🔒 Locked

HTTP Protocol

Started by rIKmAN Jan 1, 2003 at 2:17 AM 12 replies 600+ views

Original Post

Jan 1, 2003 at 2:17 AM

I am writing a program which needs to connect to a webserver POST to a PHP script. I can use GET ok and have working code to do this, but its POSTing I`m having problems with. I am sure its to do with the headers I am sending, this is the code, all line are ended with CRLF (including blank lines) A connection to the server has already been made and verified. Bear in mind that: (1) "http://www.server.com/path/to/script.php?league=2" in a browser produces the correct output. (2) The User-Agent is custom as a security measure as peopel will not be allowed to run these scripts froma browser. -- Start Code -- POST /path/to/script.php HTTP/1.0 User-Agent: CustomBrowser Accept: */* Content-type: application/x-www-form-urlencoded Content-length: 8 league=2 -- End Code -- This *should* work, but doesn`t actually display they output of the script, just the headers. Anyone have any ideas? Thanks in advance [ rIK ] [ rik@ram-solutions.co.uk ]

emptyhead

Jan 1, 2003 at 7:55 AM

i believe you can simply send:

GET /path/to/script.php?league=2 HTTP/1.0

unless the server actually expects a post, but then browsing to
http://www.server.com/path/to/script.php?league=2
wouldnt work

[edited by - emptyhead on January 1, 2003 8:56:18 AM]

CWizard

Jan 1, 2003 at 8:14 AM

Some notes:
1) Use HTTP/1.1
2) Send a "Host: somehost.com" header
3) Do not send any new lines after the post data
4) Do you use $_REQUEST[]/$_POST[] instead of $_GET[] to get the data in the php script?

EDIT:

This should work:
"POST /path/to/script.php HTTP/1.1 Host: www.server.com User-Agent: CustomBrowser Accept: */* Content-type: application/x-www-form-urlencoded Content-length: 8 league=2"

I'm also not sure about your error description. What do you mean by "just the headers". What headers are these? Bad request headers?

[edited by - CWizard on January 1, 2003 9:17:40 AM]

Themonkster

Jan 1, 2003 at 8:23 AM

have you considered using SOAP instead.

rIKmAN

Author

rIKmAN

Jan 1, 2003 at 12:56 PM

Ok guys, I have managed to fix the code, this is the working version of said code:


POST /path/to/script.cgi HTTP/1.0
User-Agent: RikBrowser
Content-Type: application/x-www-form-urlencoded
Content-Length: 8

league=2

However, I notice a SEVERE drop in speed (5-10secs) if I change from "HTTP/1.0" to "HTTP/1.1".

ANy ideas why, and does it really matter if I use "HTTP/1.0"?

Thanks

[edited by - rIKmAN on January 1, 2003 3:01:45 PM]

CWizard

Jan 1, 2003 at 3:59 PM

quote:
Original post by rIKmAN
However, I notice a SEVERE drop in speed (5-10secs) if I change from "HTTP/1.0" to "HTTP/1.1".

That may be that HTTP 1.1 defaults to Keep-Alive, where as 1.0 don''t (and don''t support it, I think). This causes the server to wait for usually 15 seconds before it close the connection so that the client can issue more requests without establishing a new connection. You can fix it by adding a header "Connection: close" which cause the server to close the connection when the response has been sent. Although, this indicates to me that your app is misbehaving somewhat

: You should close the connection yourself when you recieved the entity body.

rIKmAN

Author

rIKmAN

Jan 1, 2003 at 4:10 PM

Hehe

Yeah my game is actually in the VERY VERY early stages of development. ie. Testing ideas and stuff.

One other thing I noticed is that as part of the HTTP Protocol these headers are sent with EVERY request.
99% of the time the header info being sent will be 100-200 times the size of the actual data I need to send.

Is there any way to disable these headers being sent?
I know I`m de;ving into server-side stuff here, and Apache/PHP configuration, but I figure someone might have had these issues before.

Thanks in advance.

CWizard

Jan 1, 2003 at 4:49 PM

quote:
Original post by rIKmAN
One other thing I noticed is that as part of the HTTP Protocol these headers are sent with EVERY request.

Of course, HTTP is a stateless protocol. There is no concept such as "session" with HTTP, so you''ll need to issue a full request every time you need something.

quote:
99% of the time the header info being sent will be 100-200 times the size of the actual data I need to send.

Is there any way to disable these headers being sent?

I don''t know what you''re doing, but if you have a game or similar which constantly polls a server, HTTP is a really bad choice as protocol; sessionless and overhead. But to answer your question: no, you can only try to cut down on headers such as User-Agent, Accept. But I urge you to put in Host header, otherwise your program won''t work on a Web Server using name based virtual hosts.

rIKmAN

Author

rIKmAN

Jan 1, 2003 at 5:10 PM

Well basically I

m writing a game which will pass information to PHP Scripts, which in turn query a MySQL database, and then my game catches the output and presents it in a nice, user-friendly way.

Game -> PHP -> MySQL -> GAME

Hence why I need rid of the headers, and hence why I can

t use another protocol

Anyone?

[edited by - rIKmAN on January 1, 2003 6:11:58 PM]

CWizard

Jan 1, 2003 at 5:38 PM

How many of these messages are you sending to the server? If you send and/or request alot, I suggest you don''t use the PHP bottleneck, and code up a little server yourself. But, if you really need to you should utilize Keep-Alive, because establishing the connection will be the bottleneck that overshadows all others (unless your MySQL server is really slow, but you use PHP''s persistent connections there, huh?).

rIKmAN

Author

rIKmAN

Jan 1, 2003 at 5:45 PM

The game is going to be an online football management game, and I was going to only send data to/from the server as it was needed (no persistant connection).
This was before I found out there is no way to stop the headers being sent for EVERY connection.

I am not handling the PHP -> MySQL side of the engine, that is being done by the person working on the game with me, I am doing the front end: Game -> PHP

Would keeping a connection to the server for the duration of play (sometimes hours) be possible?

What overheads does this entail, is it a case of sending a PING request every x seconds/minutes or is it more overhead than this?
What size overhead are we talking about?

Thanks

[ rIK ]
[ rik@ram-solutions.co.uk ]

CWizard

Jan 1, 2003 at 6:03 PM

I wouldn''t worry about the header overhead unless you''re sending more than 25 requests per second. It is the packet count that counts; a packet of 10 and 500 bytes will take the same time to send, and your headers + data seems to fit nicely in a packet. To establish a new connection, 3-5 packets need to be sent between client and server.

Apache (if that is the target server) has a directive for persistent connections (Keep-Alive). Per default this is, for newer versions, 15 seconds. So, the server will drop the connection if the client does not send a request within 15 seconds. By sending a dummy request within the timeout, you can usually keep it alive. But, I suggest you use it, and let the connection be dropped if the client is idle, then reconnect when something happens. There is no real overhead; the point of persistent connections is to reduce overhead.

Another rule is to put as much data as possible into each request.

rIKmAN

Author

rIKmAN

Jan 2, 2003 at 11:03 AM

How would that work?

If I sent:

-- Start Code --
POST /path/to/script.php HTTP/1.0
User-Agent: CustomBrowser
Accept: */*
Content-type: application/x-www-form-urlencoded
Content-length: 8

league=2
-- End Code --

and the connection was still open, would I simply send:

-- Start Code --
POST /path/to/script.php HTTP/1.0

league=1
-- End Code --

as the next data, as the connection is already open.

I would think I`d have to send Content-type and Content-length headers with data even on an already established connection.

Would this be true?
If not, how would I do it and what would I need to send to an established connection?

[ rIK ]
[ rik@ram-solutions.co.uk ]

CWizard

Jan 2, 2003 at 12:05 PM

quote:
Original post by rIKmAN
would I simply send:

-- Start Code --
POST /path/to/script.php HTTP/1.0

league=1
-- End Code --

as the next data, as the connection is already open.

No. You send a new request, and so must provide all headers for the request. Remember that HTTP is a stateless protocol. The only thing that you save by using a persistent connection is that you don''t need to re-establish the connection.

quote:
I would think I`d have to send Content-type and Content-length headers with data even on an already established connection.

Would this be true?

Yes. A full request. I still don''t know the frequency of requests you are making, but I don''t think you should worry about the headers.

quote:
If not, how would I do it and what would I need to send to an established connection?

I haven''t played around with persistent HTTP connections myself, so this will be a educated guess

You should just send the request on the open socket, if it is still open, and you better make sure the server hasn''t closed it before you send. Think about it like a phone call. Without persisten connections, it''s like you dial up everytime saying what you want and hang up, over and over again. With persistent connections, you don''t need to redial all the time, but you still need to issue a full request. The server cannot assume anything about your request, as it can be (and 99% of the time is) unrelated to the previous.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.