I am using my client to get some info about a certain file stored in my Swift Object Storage which can be accessed throught REST Api. In Swift the HEAD method and url leading to specified object returns it's metadata (hash, timestamp, etc) contained in HTML response's (has NO CONTENT) headers.
My code works perfectly when file size is & 2GB. I get HttpResponseMessage and I am able to parse it for required data, but when I ask for file > 2GB I get exception : "Cannot write more bytes to the buffer than the configured maximum buffer size: ".
I understand, that HttpClient property MaxResponseContentBufferSize cannot be set to value > 2GB, but I don't want to get it's content. Is this some bug or is there some better way to solve this?
public HttpResponseMessage FileCheckResponse(string objectName)
//create url which will be appended to HttpClient (m_client)
string requestUrl = RequestUrlBuilder(m_containerName, objectName);
//create request message with Head method
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Head, requestUrl);
//exception thrown here...
HttpResponseMessage response = m_client.SendAsync(request).R
When trying to perform same action using Dev HTTP Client (Chrome extension) I have no problem. It seems that Content-Length header makes it unfeasible. Here is the output from Dev HTTP Client:
Content-Length:
Accept-Ranges: bytes
Last-Modified: Fri, 06 Sep :30 GMT
Etag: da4392bdb5c90edf31c14d
X-Timestamp: .87557
Content-Type: application/octet-stream
Date: Tue, 10 Sep :27 GMT
Connection: keep-alive
I will be glad for any ideas!
First - thanks to Darrel Mirrel, who solved my whole day problem in few seconds :)
I just needed to edit one line in code by adding HttpCompletitionOption where response is obtained:
HttpResponseMessage response = m_client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).R
The option ResponseHeaderRead tells the client to finish operation as soon as Headers are read withnout reading content of the message.
解决方案 Use the SendAsync overload that allows you to specify the HttpCompletionOptions.
With this you can tell HttpClient not to create a buffer for the response contents.
本文地址： &
我用我的客户端来获取存储在我的雨燕对象存储特定文件，可以throught REST API访问的一些信息。在斯威夫特HEAD方法和URL导致指定的对象，返回它包含在HTML响应的元数据（哈希值，时间戳等）（没有内容）的标题。
我的code完美的作品时，文件大小为＆ 2GB。我得到的Htt presponseMessage，我能够解析它所需的数据，但是当我问文件> 2GB我得到异常：“无法写入的字节数比配置的最大缓冲区大小的缓冲区：”。
据我所知，这HttpClient的财产MaxResponseContentBufferSize不能设置值> 2GB，但我不希望得到它的内容。这是一些bug或者是有一些更好的方法来解决这个问题？
公开的Htt presponseMessage FileCheckResponse（字符串对象名）
//创建URL将被追加到的HttpClient（m_client）
字符串requestUrl = RequestUrlBuilder（m_containerName，对象名）;
//创建一个头的方法，请求消息
HTT prequestMessage请求=新HTT prequestMessage（HttpMethod.Head，requestUrl）;
//异常抛出在这里...
HTT presponseMessage响应= m_client.SendAsync（要求）。结果;
当试图执行使用开发HTTP客户端（Chrome扩展）我有没有问题同样的动作。看来，Content-Length头使得它不可行。下面是从开发HTTP客户端的输出：
的Content-Length：
接受-范围：字节
最后一次修改：周五，日16时24分30秒格林尼治标准时间
ETAG：da4392bdb5c90edf31c14d
的X时间戳：.87557
内容类型：应用程序/八位字节流
日期：星期二，日13时25分27秒格林尼治标准时间
连接：保持活动
我会很高兴的任何想法！
首先 - 感谢达雷尔Mirrel，谁在几秒钟内解决了我整整一天的问题:)
我只是需要编辑code一行由那里得到的反应将HttpCompletitionOption：
的Htt presponseMessage响应= m_client.SendAsync（请求，HttpCompletionOption.ResponseHeadersRead）。结果;
选项ResponseHeaderRead告诉客户尽快头读取完成操作withnout阅读邮件的内容。
解决方案 使用的SendAsync重载，允许你指定HttpCompletionOptions。有了这个，你可以告诉HttpClient的不创建缓冲区的响应内容。
本文地址： &
扫一扫关注官方微信HttpClient使用方法
我的图书馆
HttpClient使用方法
HttpClient使用方法
本文是个人见解，有不到的地方请批评指正
HttpClient作为访问Http服务的客户端访问程序已经被广泛使用，对于HttpClient的使用方法也有很多blog进行介绍，本文简明扼要的介绍HttpClient的两种使用方式——简单连接管理的HttpClient（BasicClientConnectionManager）和池化的HttpClient（PoolingClientConnectionManager）。
HttpClient从4.2开始抛弃了先前的SingleClientConnManager和ThreadSafeConnManger，取而代之的是BasicClientConnectionManager和PoolingClientConnectionManager。
BasicClientConnectionManager内部只维护一个活动的connection，尽管这个类是线程安全的，但是最好在一个单独的线程中重复使用它。如果在同一个BasicClientConnectionManager对象中，多次执行http请求，后继请求与先前请求是同一个route，那么BasicClientConnectionManager会使用同一个连接完成后续请求，否则，BasicClientConnectionManager会将先前的connection关闭，然后为后续请求创建一个新的连接。换句话说，BasicClientConnectionManager会尽力复用先前的连接（注意：创建连接和销毁连接都是不小的开销），因此，如果对同一个service有多个连续请求，应该尽量使用同一个BasicClientConnectionManager完成。
PoolingClientConnectionManager可以在多线程中使用，连接按照route被缓存（pooled），当后续的请求route已经在pool中存在，就会使用pool中先前使用的connection获取请求结果。PoolingClientConnectionManager对每个router维护的connection数目有上限要求，默认情况下，每个router最多维护两个并发线程的connection连接，整个pool最多容纳20个并发的connections。当然可以通过设置来修改这些限制。
下面给出两个例子：
Demo 1：BasicClientConnectionManager
public static void basicClientTest() throws ClientProtocolException, IOException{
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(".cn/data/.html");
HttpResponse response = httpClient.execute(httpGet);
String result = EntityUtils.toString(response.getEntity(),
Charset.forName("utf-8"));
System.out.println(result);
httpClient.getConnectionManager().shutdown();
Demo1中，使用最简单的方式创建的httpclient内部使用的是管理连接，并且使用HttpClient提供的EntityUtils将消息体转换成String输出。httpClient对象可以被重复使用，执行多个http请求，当你真正不需要他的时候，记得调用shutdown，关闭并且释放占用的资源。
Demo2：PoolingClientConnectionManager
public static void httpclientPool()
throws ClientProtocolException, IOException{
SchemeRegistry registry = new SchemeRegistry();//创建schema
SSLContext sslContext = null;//https类型的消息访问
&&&&&&&&&&
sslContext = SSLContext.getInstance("SSL");
&&&&&&&&&&
sslContext.init(null, null, null);
catch (Exception e) {
&&&&&&&&&&
e.printStackTrace();
SSLSocketFactory sslFactory = newSSLSocketFactory(sslContext,SSLSocketFactory.STRICT_HOSTNAME_VERIFIER);
registry.register(new Scheme("http",
PlainSocketFactory.getSocketFactory()));//http 80 端口
registry.register(new Scheme("https",
443, sslFactory));//https 443端口
PoolingClientConnectionManager cm = newPoolingClientConnectionManager(registry);//创建connectionManager
cm.setDefaultMaxPerRoute(20);//对每个指定连接的服务器（指定的ip）可以创建并发20
socket进行访问
cm.setMaxTotal(200);//创建socket的上线是200
HttpHost localhost = new HttpHost("locahost",
cm.setMaxPerRoute(new HttpRoute(localhost), 80);//对本机80端口的socket连接上限是80
HttpClient httpClient = new DefaultHttpClient(cm);//使用连接池创建连接
HttpParams params = httpClient.getParams();
HttpConnectionParams.setSoTimeout(params,
60*1000);//设定连接等待时间
HttpConnectionParams.setConnectionTimeout(params,
60*1000);//设定超时时间
&&&&&&&&&&
HttpGet httpGet = new HttpGet(".cn/data/.html");
&&&&&&&&&&
HttpResponse response = httpClient.execute(httpGet);
&&&&&&&&&&
String result = EntityUtils.toString(response.getEntity(),
Charset.forName("utf-8"));
&&&&&&&&&&
System.out.println(result);
&&&&&&&&&&
httpClient.getConnectionManager().shutdown();//用完了释放连接
PoolingClientConnectionManager的创建过程要比BasicClientConnectionManager复杂的多，当然，我们为了举例子，将PoolingClientConnectionManager可能涉及到的多个方面属性都列出来，可以根据需要设置或者使用默认的连接属性。首先我们创建了SchemeRegistry，之后注册了两种http访问方式http和https，创建，指定每个router最多保持20个活动的connection，这个pool最多保持200个活动的connection，对本地的访问可以保持80个活动的connection。最后创建httpClient，设置SO_TIMEOUT和CONNECTION_TIMEOUT，执行请求。在httpClient不再使用时，关闭connectionManager。
只是个例子，说明应当如何创建一个带有缓冲池的HttpClient，在实际应用中，可以对多个访问共用一个池化的HttpClient，即提高了性能（重用connection，connection和disconnect是非常耗时的，要选路由，三次握手等），也方便了管理，有利于代码的维护。
TA的推荐TA的最新馆藏
喜欢该文的人也喜欢HttpClient学习研究---第六章：HTTP缓存 - CSDN博客
HttpClient学习研究---第六章：HTTP缓存
第六章。HTTP CachingHTTP缓存
6.1。General Concepts一般概念
HttpClient Cache provides an HTTP/1.1-compliant caching layer to be used with HttpClient--the Java equivalent of a browser cache.
HttpClient缓存提供了一个HTTP / 1.1兼容的缓存层可以使用HttpClient——Java相当于一个浏览器缓存。The implementation follows the Chain of Responsibility design pattern, where the caching HttpClient implementation can serve a drop-in replacement for the
default non-caching HttpC requests that can be satisfied entirely from the cache will not result in actual origin requests.
实现遵循责任链设计模式,实现服务的缓存HttpClient顺便替代了默认非缓存HttpClient实施;请求,可以满足完全从缓存将不会导致实际来源请求。Stale cache entries are automatically validated with the origin where possible, using conditional GETs and the If-Modified-Since and/or If-None-Match
request headers.过期的缓存条目自动验证的原产地在可能的情况下,使用条件会和如果以后修改和/或没有请求头。
HTTP/1.1 caching in general is designed to beHTTP / 1.1??存通常被设计成
semantically transparent语义透明; that is, a cache should not change the meaning of the request-response exchange between client and server.
,即,一个缓存应该没有改变的意思-响应客户机和服务器之间交换。As such, it should be safe to drop a caching HttpClient into an existing compliant client-server relationship.
因此,它应该是安全滴一个缓存HttpClient到现有的客户机-服务器兼容的关系。Although the caching module is part of the client from an HTTP protocol point of view, the implementation aims to be compatible with the requirements placed on a transparent caching
proxy.尽管缓存模块部分的客户从HTTP协议的角度来看,实施旨在适合需求放在一个透明的缓存代理。
Finally, caching HttpClient includes support the Cache-Control extensions specified by RFC 5861 (stale-if-error and stale-while-revalidate).最后,缓存HttpClient包括支持指定的cache - control扩展RFC 5861(过期如果误差和陈旧而重新验证)。
When caching HttpClient executes a request, it goes through the following flow:当缓存HttpClient执行一个请求时,它会通过以下流程:
Check the request for basic compliance with the HTTP 1.1 protocol and attempt to correct the request.检查要求基本符合HTTP 1.1协议并尝试正确的请求。
Flush any cache entries which would be invalidated by this request.平任何缓存条目将无效,因为这个请求。
Determine if the current request would be servable from cache.
确定当前请求从缓存将违法者给予。If not, directly pass through the request to the origin server and return the response, after caching it if appropriate.如果没有,直接通过请求源服务器并返回响应,在缓存它如果适当的。
If it was a a cache-servable request, it will attempt to read it from the cache.
如果这是一个一个缓存违法者给予请求时,它将尝试从缓存中读取它。If it is not in the cache, call the origin server and cache the response, if appropriate.如果它不是在缓存中,调用源服务器和缓存响应,如果适当的。
If the cached response is suitable to be served as a response, construct a BasicHttpResponse containing a ByteArrayEntity and return it.
如果缓存的响应是适合作为响应,构建一个包含一个ByteArrayEntity BasicHttpResponse并返回它。Otherwise, attempt to revalidate the cache entry against the origin server.否则,尝试重新验证缓存条目与源服务器。
In the case of a cached response which cannot be revalidated, call the origin server and cache the response, if appropriate.对于一个缓存的响应不能重新检验它,叫源服务器和缓存响应,如果适当的。
When caching HttpClient receives a response, it goes through the following flow:当缓存HttpClient收到一个响应,它会通过以下流程:
Examining the response for protocol compliance检查响应协议遵从性
Determine whether the response is cacheable确定是否可缓存的响应
If it is cacheable, attempt to read up to the maximum size allowed in the configuration and store it in the cache.如果它是缓存,试图读到最大尺寸允许的配置并将其存储在缓存中。
If the response is too large for the cache, reconstruct the partially consumed response and return it directly without caching it.如果响应是太大的缓存,重建部分响应并返回它直接消耗没有缓存它。
It is important to note that caching HttpClient is not, itself, a different implementation of HttpClient, but that it works by inserting itself as an additonal processing component to the request execution pipeline.重要的是要注意,缓存不是HttpClient,本身,一个不同的实现,但这作品HttpClient通过插入本身作为一个额外的处理组件请求执行管道。
6.2。RFC-2616 Compliancerfc - 2616合规
We believe HttpClient Cache is我们相信HttpClient缓存
unconditionally compliant无条件顺从的with与. 。That is, wherever the specification indicates MUST, MUST NOT, SHOULD, or SHOULD NOT for HTTP caches, the caching layer attempts to behave in a way that satisfies those requirements.
即,无论该规范表明必须,必须,应该或不应该对于HTTP缓存,缓存层试图表现的方式满足这些需求。This means the caching module won't produce incorrect behavior when you drop it in.这意味着缓存模块不会产生不正确的行为,当你把它在。
6.3。Example Usage示例使用
This is a simple example of how to set up a basic caching HttpClient.
这是一个简单的例子,如何建立一个基本的缓存HttpClient。As configured, it will store a maximum of 1000 cached objects, each of which may have a maximum body size of 8192 bytes.
根据配置,它将存储的最大值是1000缓存对象,每一种都可能有一个最大的身体大小为8192字节。The numbers selected here are for example only and not intended to be prescriptive or considered as recommendations.数字选择这里有例如只有和不准备规定性或视为建议。
CacheConfig cacheConfig = CacheConfig.custom()
.setMaxCacheEntries(1000)
.setMaxObjectSize(8192)
RequestConfig requestConfig = RequestConfig.custom()
.setConnectTimeout(30000)
.setSocketTimeout(30000)
CloseableHttpClient cachingClient = caching HttpClients.custom()
.setCacheConfig(cacheConfig)
.setDefaultRequestConfig(requestConfig)
HttpCacheContext context = HttpCacheContext.create();
HttpGet httpget = new HttpGet(&/content/&);
CloseableHttpResponse response = cachingClient.execute(httpget, context);
CacheResponseStatus responseStatus = context.getCacheResponseStatus();
switch (responseStatus) {
case CACHE_HIT:
System.out.println(&A response was generated from the cache with & +
&no requests sent upstream&);
case CACHE_MODULE_RESPONSE:
System.out.println(&The response was generated directly by the & +
&caching module&);
case CACHE_MISS:
System.out.println(&The response came from an upstream server&);
case VALIDATED:
System.out.println(&The response was generated from the cache & +
&after validating the entry with the origin server&);
} finally {
response.close();
6.4。Configuration配置
The caching HttpClient inherits all configuration options and parameters of the default non-caching implementation (this includes setting options like timeouts and connection pool sizes).
缓存HttpClient继承所有的配置选项和参数的默认非缓存实现(这包括设置选项如超时和连接池大小)。For caching-specific configuration, you can provide a对缓存的具体配置,您可以提供一个 CacheConfiginstance to customize
behavior across the following areas:实例来定制行为在以下领域:
Cache size.缓存大小。If the backend storage supports these limits, you can specify the maximum number of cache entries as well as the maximum cacheable
response body size.如果后端存储支持这些限制,你可以指定缓存条目的最大数量以及最大缓存响应体大小。
Public/private caching.公共/私有缓存。By default, the caching module considers itself to be a shared (public) cache, and will not, for example, cache responses
to requests with Authorization headers or responses marked with &Cache-Control: private&.
默认情况下,缓存模块认为自己是一个共享的(公共)缓存,不会,例如,缓存响应或响应请求的授权标题注明“cache - control:私人”。If, however, the cache is only going to be used by one logical &user& (behaving similarly to a browser cache), then you will want to turn off the shared
cache setting.然而,如果缓存中只会使用一个逻辑上的“用户”(同样的行为到浏览器缓存),然后你会想关掉共享缓存设置。
Heuristic caching.启发式缓存。Per RFC2616, a cache MAY cache certain cache entries even if no explicit cache control headers are set by the origin.
每RFC2616,缓存可能缓存特定的缓存条目即使没有显式的缓存控制标题设置的起源。This behavior is off by default, but you may want to turn this on if you are working with an origin that doesn't set proper headers but where you still want to cache the responses.
这种行为默认是关闭的,但是你可能想要把这个如果你正与一个起源,没有设置适当的标题,但你仍然想要缓存的响应。You will want to enable heuristic caching, then specify either a default freshness lifetime and/or a fraction of the time since the resource was last modified.
你会想让启发式缓存,然后指定一个默认的新鲜要么一生和/或一个短的时间自资源最后修改。See Sections 13.2.2 and 13.2.4 of the HTTP/1.1 RFC for more details on heuristic caching.见章节13 2 2和13 2 4 HTTP / 1.1 RFC详情启发式缓存。
Background validation.背景验证。The cache module supports the stale-while-revalidate directive of RFC5861, which allows certain cache entry revalidations
to happen in the background. 缓存模块支持的RFC5861陈旧而重新验证指令,允许特定的缓存条目revalidations发生的背景。You may want to tweak the settings for the minimum and maximum number of background worker threads, as well as the maximum time they can
be idle before being reclaimed. 你可能想要调整设置最小和最大数量的背景工作线程,以及最大的时间他们可以被回收之前闲置。You can also control the size of the queue used for revalidations when there aren't enough workers to keep up with demand.您还可以控制队列的大小用于revalidations当没有足够的工人,才能跟上需求。
6.5。Storage Backends存储后端
The default implementation of caching HttpClient stores cache entries and cached response bodies in memory in the JVM of your application.
默认实现缓存的缓存条目和HttpClient存储在内存中缓存的响应的身体在JVM您的应用程序。While this offers high performance, it may not be appropriate for your application due to the limitation on size or because the cache entries are ephemeral and don't survive
an application restart. 虽然这提供高性能,它可能不适合您的应用程序由于限制大小或因为缓存条目都是短暂的,不要在应用重启。The current release includes support for storing cache entries using EhCache and memcached implementations, which allow for spilling cache entries
to disk or storing them in an external process.当前版本支持存储缓存条目使用EhCache和memcached的实现,它允许溢出到磁盘缓存条目或将它们存储在外部过程。
If none of those options are suitable for your application, it is possible to provide your own storage backend by implementing the HttpCacheStorage interface and then supplying that to caching HttpClient at construction time.
如果没有这些选项适合您的应用程序,可以提供自己的存储后端通过实现接口,然后HttpCacheStorage供应,在构造时,缓存HttpClient。In this case, the cache entries will be stored using your scheme but you will get to reuse all of the logic surrounding HTTP/1.1 compliance and
cache handling. 在这种情况下,缓存条目将存储使用您的方案,但你会得到重用所有的逻辑周围的HTTP / 1.1合规和缓存处理。Generally speaking, it should be possible to create an HttpCacheStorage implementation out of anything that supports a key/value store (similar to
the Java Map interface) with the ability to apply atomic updates.一般来说,它应该有可能创建一个HttpCacheStorage实现从任何支持一键/值存储(类似于Java Map接口)与原子更新应用能力。
Finally, with some extra efforts it's entirely possible to set up a multi-ti for example, wrapping an in-memory caching HttpClient around one that stores cache entries on disk or remotely in memcached,
following a pattern similar to virtual memory, L1/L2 processor caches, etc.最后,一些额外的努力完全有可能建立一个多层缓存层次;例如,包装一个内存中的缓存HttpClient围绕着一个高速缓存条目存储在磁盘或远程在memcached,遵循一个模式类似于虚拟内存,L1 / L2处理器缓存等。
本文已收录于以下专栏：
相关文章推荐
看到很多小伙伴在聊OkHttp的缓存问题，用好OkHttp缓存我们可以少写很多代码。
Caller类对HttpURLConnection和HttpClient两种网络访问机制的Post和Get请求都进行了封装,并且加入了全局缓存机制.需要使用缓存机制必须加入RequestCache类,...
html ,js 可以实现页面跳转。
jsp , asp, php 也有各自页面跳转与重定向的方式。
下文针对js 和jsp 的页面跳转实现方式进行一个总结。
&?ob_start();……..if ( something ){     ob_end_clean();     header(“Location: yourlocation”)；  &#1...
首先说说HttpClient和浏览器的区别
       HttpClient只是模拟一次tcp请求，浏览器则会帮你处理重定向、缓存等事情。这也就是为什么用浏览器表单post提交后，不管服务端如何重定...
《用Apache HttpClient实现URL重定向》作者：chszs，转载需注明。博客主页：http://blog.csdn.net/chszs很多网站都使用了URL重定向技术，把一个原始请求从一...
http头信息
头信息的作用很多，最主要的有下面几个：
当浏览器接受到头信息中的 Location: xxxx 后，就会自动跳转到 xxxx 指向的URL地址，这点有点类似用 js...
Xe8提供了THttpClient,发送Web请求就简单多了!
我用他下载一个文件，代码变的非常简洁。
XE8 用HttpClient下载文件& title=&Delphi XE8 用HttpCl...
第四章。HTTP authenticationHTTP身份验证
HttpClient provides full support for authentication schemes d...
第三章。HTTP state managementHTTP状态管理
Originally HTTP was designed as a stateless, request / resp...
他的最新文章
讲师：吴岸城
您举报文章：
举报原因：
原文地址：
原因补充：
(最多只允许输入30个字)