🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Linux sockets + http get request = no data

Started by
15 comments, last by hplus0603 4 years, 10 months ago

Currently.i need to fetch some data from a web page but i cant get anything from it. Reading through internet all samples provide something like this:

- connect socket to server with port 80

- write http request

- read back data

And they all say it works, but well not for me..

Heres a sample code i use - i use a class from my networking engine so it should work.

 


int main()
{
	TCPWindowLayerClient * c = new TCPWindowLayerClient();
	c->ConnectTo("http://www.google.com", 80);
	SendStrToFd( "GET / HTTP/1.1\r\nHost: http://www.google.com\r\nConnection: close\r\n\r\n", c->sockfd);
	int * ppos = new int;
	bool breakme = false;
	while ( !breakme )
	{
		int nbytes;
		 (*ppos) = 0;
	AnsiString r =	read_from_fd2(c->sockfd, c->pdata, ppos, max_tcp_buff_size, nbytes);
breakme = (nbytes <= 0);
		cout << r.c_str();
	}
	
	cout << "end.";
	return 0;
}

Only end. Is displayed and errno is 107 which means connection is closed.

Advertisement

Some suggestions:

(1) Don't use 'Connection:close" in the header (shouldn't hurt, but why complicate things when troubleshooting?)

(2) You're using some kind of mysterious third-party library. Start by consulting the documentation on the library.  Since the problem appears to be either your use of the library or the library itself, and you have posted nothing about the library including its name and where someone else can find information on it, it's not possible to offer any kind of help about your problem here.

Stephen M. Webb
Professional Free Software Developer

sendstrtofd is just a write function, (        int wb = write(sockfd, &pdata[ count ], size_t( len_to_write ));)

 

readfromfd2 is just a read function. 

(  nbytes = read (sockfd, buffer, TCP_PROTOCOL_MAX_PACKET_LEN);)

 

Nothing more.

Connection is done like this:


	    int ConnectTo(AnsiString serv, int port)
	    {

Clear();

	    	 portno = port;
	    	    sockfd = socket(AF_INET, SOCK_STREAM, 0);

	    	    if (sockfd < 0) return -1;

	    	    server = gethostbyname(serv.c_str());

	    	    if (server == NULL) {
	    	    	LOG("serve null no host");
	    	       return -1;
	    	    }

	    	    bzero((char *) &serv_addr, sizeof(serv_addr));
	    	    serv_addr.sin_family = AF_INET;

	    	    serv_addr.sin_addr.s_addr = INADDR_ANY;
	    	    bcopy((char *)server->h_addr,
	    	         (char *)&serv_addr.sin_addr.s_addr,
	    	         server->h_length);
	    	    serv_addr.sin_port = htons((uint16_t)portno);

	    	    if (connect(sockfd,(struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0)
	    	         	{
	    	         		LAST_EVENT = TCP_ERROR;
	    	    	
	    	    	return -1;
	    	         	}
	    	         	Clear();
	    	         	timer.startTimer();
	    	         
return 1;
	    }

Also about this header connection:close it was just a sample i downloaded from internet, with or without it the result is the same (no data and this errno 107)

If you're on linux, use Wireshark to watch what is going over the wire.  If the response is coming in to your machine, but you're not receiving it, the problem is when you read the socket.  If you're not getting a response back at all, the problem is elsewhere.

Use telnet to connect to the remote and and emulate what you think your program is doing, while watching the Wireshark output.  See if they're different.

Stephen M. Webb
Professional Free Software Developer

Unfortunatetly theres error on connection (cant even connect) now im not quite aware if i should set ip address or can i use www.google.com, maybe theres additional option for the socket that i do not set?

For comparison, this connects. Maybe it is useful, maybe not ?


// @todo check errno
int fd = socket( AF_INET, SOCK_STREAM, 0 );
if( fd > 0 )
	fputs( "Socket created\n", stdout );

struct sockaddr_in addr;
//memset( &addr, 0, sizeof( addr ) );
addr.sin_family = AF_INET;
addr.sin_port = htons( 80 );
if( 0 == inet_aton( "13.32.90.60", &addr.sin_addr ) )	// www.n-tv.de
	fputs( "Error converting address\n", stderr );
else
	fputs( "Address converted\n", stdout );

if( 0 != connect( fd, (struct sockaddr *)&addr, sizeof( addr ) ) )
	fputs( "Error trying to connect\n", stderr );
else
	fputs( "Connection established\n", stdout );

// ... do somehting ....

 

Gethostbyname returns null

That's bad.

Here's a plan b for converting the inet address to something connectible that connects. I love that tinkering ?


int fd = socket( AF_INET, SOCK_STREAM, 0 );
if( fd > 0 )
	fputs( "Socket created\n", stdout );

struct hostent *s = gethostbyname( "www.n-tv.de" );
if( NULL == s )
	fputs( "Address lookup failed\n", stderr );

struct in_addr **r = s->h_addr_list;

struct sockaddr_in addr;
//memset( &addr, 0, sizeof( addr ) );
addr.sin_family = AF_INET;
addr.sin_port = htons( 80 );
addr.sin_addr = *r[0];

if( 0 != connect( fd, (struct sockaddr *)&addr, sizeof( addr ) ) )
	fputs( "Error connecting to socket\n", stderr );
else
	fputs( "Connection established\n", stdout );

 

in my eample gethostbyname was returning NULL (just to clarify)

in your first example i get error converting address

and in second one i cant compile this line 


struct in_addr **r = s->h_addr_list;

however i see that you use ip address instead of url

 

Additionally if i try to resolve www.google.com to ip address errno claims that HOST IS NOT FOUND....

1 hour ago, _WeirdCat_ said:

in my eample gethostbyname was returning NULL (just to clarify)

Yes, because gethostname expects a hostname, not an url. man gethostbyname helps, but also says it is deprecated and getnameinfo getaddrinfo() should be used instead. But some finger training won't harm ?

Quote

in your first example i get error converting address

That example works on my machine. Something is probably borked and needs further unborking. The exact code might help.

Quote

and in second one i cant compile this line 



struct in_addr **r = s->h_addr_list;

however i see that you use ip address instead of url

Yeah, it results in a warning. It is a working example. Explicitly casting would be better. But you can use the pointer to hike along the list of ip addresses, convert them and print them out.


while( *r != NULL ) {
	fputs( inet_ntoa( **r ), stdout );
	putchar( '\n' );
	++r;
}

 

Quote

 

Additionally if i try to resolve www.google.com to ip address errno claims that HOST IS NOT FOUND....

No idea. I pasted that code directly from the editor after it ran. If you resolve "https://..." you should get that error, "www.google.com" should work ...

 

 

This topic is closed to new replies.

Advertisement