Zipping HTTP Request

Discussion:

Philippe Mouawad

2018-05-07 15:15:26 UTC

Hello,
There are sometimes a requirement to automatically gzip the request body
whenever Content-Encoding is set to gzip.

Would you find it interesting to add this feature to HttpClient ?

The implementation would look like:

private static final class GzipHttpRequestInterceptor implements
HttpRequestInterceptor {
@Override
public void process(HttpRequest request, HttpContext context)
throws HttpException, IOException {
if(request instanceof HttpEntityEnclosingRequest) {
Header header = request.getFirstHeader("Content-Encoding");
if(header != null && "gzip".equals(header.getValue())) {
HttpEntityEnclosingRequest enclosingRequest =
(HttpEntityEnclosingRequest) request;
HttpEntity entity = enclosingRequest.getEntity();
ByteArrayOutputStream out = new ByteArrayOutputStream();
try (GZIPOutputStream gzipOS = new
GZIPOutputStream(out)) {
entity.writeTo(gzipOS);
}
enclosingRequest.setEntity(new
ByteArrayEntity(out.toByteArray()));
}
}
}
}

Use could add it to processing like this:

- builder.addInterceptorFirst(new GzipHttpRequestInterceptor() );

Or it could be enabled by default.

Questions:

- Would you accept it as a PR ?
- Do you see immediate improvements in the provided code ?

Thanks

--
Regards
Philippe Mouawad.
Ubik-IngÃ©nierie

Gary Gregory

2018-05-07 15:24:01 UTC

Permalink

What could be more useful is a more general solution in a new module based
on Apache Commons Compress that calls
org.apache.commons.compress.compressors.CompressorStreamFactory.createCompressorOutputStream(String,
OutputStream).

That would give you support for:

GZIP
BZIP2
XZ
PACK200
LZMA
DEFLATE
SNAPPY_FRAMED
LZ4_BLOCK
LZ4_FRAMED
ZSTANDARD

Gary

On Mon, May 7, 2018 at 9:15 AM, Philippe Mouawad <

Post by Philippe Mouawad
Hello,
There are sometimes a requirement to automatically gzip the request body
whenever Content-Encoding is set to gzip.
Would you find it interesting to add this feature to HttpClient ?
private static final class GzipHttpRequestInterceptor implements
HttpRequestInterceptor {
@Override
public void process(HttpRequest request, HttpContext context)
throws HttpException, IOException {
if(request instanceof HttpEntityEnclosingRequest) {
Header header = request.getFirstHeader("
Content-Encoding");
if(header != null && "gzip".equals(header.getValue())) {
HttpEntityEnclosingRequest enclosingRequest =
(HttpEntityEnclosingRequest) request;
HttpEntity entity = enclosingRequest.getEntity();
ByteArrayOutputStream out = new
ByteArrayOutputStream();
try (GZIPOutputStream gzipOS = new
GZIPOutputStream(out)) {
entity.writeTo(gzipOS);
}
enclosingRequest.setEntity(new
ByteArrayEntity(out.toByteArray()));
}
}
}
}
- builder.addInterceptorFirst(new GzipHttpRequestInterceptor() );
Or it could be enabled by default.
- Would you accept it as a PR ?
- Do you see immediate improvements in the provided code ?
Thanks
--
Regards
Philippe Mouawad.
Ubik-IngÃ©nierie

Gary Gregory

2018-05-07 17:47:22 UTC

Permalink

Hi,

Brotli is supported in read-only mode:
https://commons.apache.org/proper/commons-compress/limitations.html

This is a limitation of the underlying library org.brotli.dec.

We welcome patches ;-)

Gary

On Mon, May 7, 2018 at 10:50 AM, Christopher Schultz <

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Gary,

Post by Gary Gregory
What could be more useful is a more general solution in a new
module based on Apache Commons Compress that calls
org.apache.commons.compress.compressors.CompressorStreamFactory.create

CompressorOutputStream(String,
OutputStream).

Post by Gary Gregory
GZIP BZIP2 XZ PACK200 LZMA DEFLATE SNAPPY_FRAMED LZ4_BLOCK
LZ4_FRAMED ZSTANDARD

What, no Brotli[1]? ;)
I'm almost entirely kidding, but most of the above compression methods
are not supported by web servers. RFC 7231 only defined
content-encoding tokens for gzip, deflate, and compress/x-compress
(and "identity"). Mozilla extends that list to include 'br' (for Brotli)
.
So, while using a generalized method for compression, you might want
to limit it to actually-useful compression types.
- -chris
[1] https://en.wikipedia.org/wiki/Brotli

Post by Gary Gregory
On Mon, May 7, 2018 at 9:15 AM, Philippe Mouawad <

Hello, There are sometimes a requirement to automatically gzip
the request body whenever Content-Encoding is set to gzip.
Would you find it interesting to add this feature to HttpClient ?
private static final class GzipHttpRequestInterceptor implements
process(HttpRequest request, HttpContext context) throws
HttpException, IOException { if(request instanceof
HttpEntityEnclosingRequest) { Header header =
request.getFirstHeader(" Content-Encoding"); if(header != null &&
"gzip".equals(header.getValue())) { HttpEntityEnclosingRequest
enclosingRequest = (HttpEntityEnclosingRequest) request;
HttpEntity entity = enclosingRequest.getEntity();
ByteArrayOutputStream out = new ByteArrayOutputStream(); try
(GZIPOutputStream gzipOS = new GZIPOutputStream(out)) {
entity.writeTo(gzipOS); } enclosingRequest.setEntity(new
ByteArrayEntity(out.toByteArray())); } } } }
- builder.addInterceptorFirst(new GzipHttpRequestInterceptor() );
Or it could be enabled by default.
- Would you accept it as a PR ? - Do you see immediate
improvements in the provided code ?
Thanks -- Regards Philippe Mouawad. Ubik-IngÃ©nierie