Tips to Handle Request/Response Size

In general, web servers implementing the HTTP 1.1 protocol do not place any prior limit to the length of characters in the URI and post requests. So practically all the modern browsers built on HTTP 1.1 should not restrict any length of data, but servers implementing this protocol restrict the data size if the URI size is more than 255 bytes.

In all practical purposes, this solution is tested in Akamai (as a CDN), Apache (as a Proxy/Reverse Proxy/Web server) and Adobe-AEM (as a CMS). But the solutions are generic in nature and can be applied to individual setups or combined as a whole.

Servers like Apache Web server, Akamai, and Adobe AEM are only allowing 8KB of data in request and response. If the data size is more than the allowed length, we should get an error "400- Bad Request" or "414 - Request-URI Too Long."

If you are getting 400- Bad Request this is because the server has a set limit in either the proxy or reverse proxy; whereas, if we get 414 - Request-URI Too Long it means the URI size needs to be increased.

Here are some server specific use cases/solutions to 400- Bad Request and 414 -Request-URI Too Long.

Use Case 1: 400-Bad Request - Adobe (AEM)

If you're running a request/response which involves a large amount of data transfer through the AEM publisher, then increase the default size to 8KB of data, otherwise, you will get a 400-Bad Request.

The fix depends on versions of AEM/CQ.

If the version is 5.6 or lower then use the following steps:

1. Go to https://www.xxxx.com/system/console/configMgr

2. Search CQSE and click on the link to the Day CQSE HTTP Service.

3. A new pop-up window will be opened. Change the value of "Request Buffer Size" to the size you need (the default is 8KB).

4. Click 'Save.' AEM/CQ can now handle a larger amount of data than what you had originally set up. Refer to the picture of AEM/CQ 5.X below. 

                                                                   AEM/CQ 5.X

If the version is 6.x or higher, then use the following steps:

1. Go to https://www.xxxx.com/system/console/configMgr

2. Search “Apache Felix Jetty Based Http Service"; then click on the link “Apache Felix Jetty Based Http Service." 

3. In 6.2, the solution moved to Felix jetty and jetty provides two fields “Header Buffer Size” and “Request Buffer Size.” The “Header Buffer Size” provides 16KB by default and “Request Buffer Size” provides 8KB. So, if the total request size is more than 24 KB, then increase “Request Buffer Size.”

4. Click 'Save.' AEM/CQ can now handle a larger amount of data. Refer to the picture AEM/CQ 6.X below: 

                                                                  AEM/CQ 6.X

Use Case 2: 400- Bad Request - Apache Dispatcher-Adobe (CQ) Integration

If you get a request/response which involves a large amount of data in an Apache(Dispatcher)-AEM/CQ integration scenario, AEM/CQ dispatcher has a size limit that all the URLs go through. Adobe provides a dispatcher that says it has the fix to handle more than 8 KB, but it did not work for us. To fix this:

2.1:

Remove the URL pattern from dispatcher.any so that that request does not go thru dispatcher/filter.

{

#/0104 { /type "allow" /glob "* /hreq/myapi/*" }

/0041 { /type "allow" /glob "* *.css *" } # enable css

/0042 { /type "allow" /glob "* *.gif *" } # enable gifs

/0043 { /type "allow" /glob "* *.ico *" } # enable icos

}

Make sure to note that #104 says any request with the pattern hreq/myapi will not be allowed through the dispatcher.

2.2:

Write the proxy rules so they can directly access the publisher URL instead of going through the dispatcher:

RewriteCond %{REQUEST_URI} ^/hreq/myapi/(.*)$
RewriteRule ^/hreq/myapi/(.*)$ http://xxxxxx:6503/hreq/myapi/$1 [P,L,QSA]

Here, any incoming request to Apache will directly go to the IP (xxxxx:6503) for any request that comes for "hreq/myapi" and will not go through the dispatcher. This can be mapped to domain URLs if you have that set up in the prod server like so: 

RewriteCond %{REQUEST_URI} ^/hreq/myapi/(.*)$
RewriteRule ^/hreq/myapi/(.*)$ http://mydomain.com/hreq/myapi/$1 [P,L,QSA]

We need both steps from Use Case 2 (2.1 and 2.2 to be fixed. Fixing one step will not resolve the issue.

Use Case 3: 400- Bad Request – Akamai

If your domain is hosted in Akamai then your request/response will not be processed because Akamai has a limit it's header size to 8KB. If you get the error 400 Bad in the browser and Akamai tells you the reason is Request "R_FWD_BAD_HEADERSInvalidForwardReply: The forward reply headers were malformed or too large,” then the error will occur when the HTTP response status line is missing or malformed or when the forward response headers exceed network:http.response-headers.max-size which defaults to 8192 bytes. Then it is clear we need to increase the header size in Akamai. 

Use Case 4: 414 - Request-URI Too Long - Apache

If your URI contains a large amount of data then Apache will return 414-Request-URI Too Long. The fix is in Apache config file(httpd.conf) is to increase the size by adding the following line:

LimitRequestLine 16384 

This will increase the size to 16KB. If you set LimitRequestLine to 0 then the size will be 2GB and that will be the maximum allowed size. The default size is 8KB.