请求同一个网址， requests 和 urllib2 返回的 headers 内容不同？

Date: XXX Server: Apache Last-Modified: XXX Accept-Ranges: bytes Content-Length: 12345678 Vary: Accept-Encoding Connection: close Content-Type: text/plain

requests:

Content-Encoding: gzip Accept-Ranges: bytes Vary: Accept-Encoding Keep-alive: timeout=5, max=128 Last-Modified: XXX Connection: Keep-Alive ETag: xxxxxxxxx Content-Type: text/plain

为何 requests 少了 content-length ？其它发送请求的设置是完全一样的。。 requests 和 Chrome 开发者工具查看到的一致。但是这里我又需要 content-length 的值（为了断点续传）

第 1 条附言 2017-01-14 17:56:30 +08:00

import urllib2 import requests url = 'exmaple.com' headers = { "Authorization": "Basic xxxx", "Range": "bytes=0-" } req = urllib2.Request(url, headers=headers) resp = urllib2.urlopen(req) print resp.info() r = requests.get(url, headers=headers) print r.headers assert resp.info()['ETag'] == r.headers['ETag']

Date: Sat, 14 Jan 2017 09:39:50 GMT Server: Apache Last-Modified: Sat, 14 Jan 2017 09:39:49 GMT ETag: "e91103-10e04f7-5460abb4743a3" Accept-Ranges: bytes Content-Length: 17695991 Vary: Accept-Encoding Content-Range: bytes 0-17695990/17695991 Connection: close Content-Type: text/plain {'Content-Encoding': 'gzip', 'Transfer-Encoding': 'chunked', 'Accept-Ranges': 'bytes', 'Vary': 'Accept-Encoding', 'Keep-Alive': 'timeout=5, max=128', 'Server': 'Apache', 'Last-Modified': 'Sat, 14 Jan 2017 09:39:49 GMT', 'Connection': 'Keep-Alive', 'ETag': '"e91103-10e04f7-5460abb4743a3"', 'Date': 'Sat, 14 Jan 2017 09:39:50 GMT', 'Content-Type': 'text/plain'}

我也知道肯定是两次发送的请求header不一样。。。现在总算解决了。。

第 2 条附言 2017-01-14 17:57:18 +08:00

The response is different because requests indicates that it supports gzip-encoded bodies, by sending an Accept-Encoding: gzip, deflate header field. urllib2 does not. You'll find if you added that header to your urllib2 request that you get the new behaviour.

Clearly, in this case, the server is dynamically gzipping the responses. This means it doesn't know how long the response will be, so it is sending using chunked transfer encoding.

If you really must get the Content-Length header, then you should add the following headers to your Requests request: {'Accept-Encoding': 'identity'}.

15 条回复 2017-01-14 23:11:08 +08:00

redhatping

2017-01-14 17:18:47 +08:00

看官方文档

binux

2017-01-14 17:20:43 +08:00 via Android

Content-Length 不应该手动设置

dofine

2017-01-14 17:24:10 +08:00

@binux 我描述不清～～上边给出的结果是响应的 header ，我的意思是需要知道当前 content-length 的值。。但是 requests 的返回里面没有。。 urllib2 就有。。
@redhatping 文档已经看了许多遍了。。怀疑是服务器的问题？

hahastudio

2017-01-14 17:25:47 +08:00

你这是结果，肯定还是因为你发送的请求不一样

Lonely

2017-01-14 17:36:14 +08:00 via iPhone

你把代码也贴出来啊……

dofine

2017-01-14 17:37:59 +08:00

{'Range': 'bytes=0-', 'Authorization': 'Basic XXX'}

手动加了这个 header ， urllib2 和 requests 返回的 ETag 都是一样的啊。。为什么会发送请求不一样呢。。 @hahastudio

dofine

2017-01-14 17:40:06 +08:00

```
import os
import urllib2
import requests

url = 'exmaple.com'
headers = {
"Authorization": "Basic xxxx",
"Range": "bytes=0-"
}
req = urllib2.Request(url, headers=headers)
resp = urllib2.urlopen(req)
print resp.info()

r = requests.get(url, headers=headers)
print r.headers
assert resp.info()['ETag'] == r.headers['ETag']
```

dofine

2017-01-14 17:42:28 +08:00

@Lonely 贴在楼上了。。我发现我都不在回复里贴代码了。。