Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

HTTP/2 With Server Push Proof of Concept for ASP.NET Core HttpSysServer

DZone's Guide to

HTTP/2 With Server Push Proof of Concept for ASP.NET Core HttpSysServer

In this article, we take a stab at solving the problem of getting an HTTP/2 with a server push from our ASP.NET Core application that we've hosted on an HttpSysServer.

· Web Dev Zone
Free Resource

Add user login and MFA to your next project in minutes. Create a free Okta developer account, drop in one of our SDKs to your application and get back to building.

Recently I've been playing a lot with HTTP/2 and with ASP.NET Core but I didn't have a chance to play with both at once. I've decided it's time to change that. Unfortunately, the direct HTTP/2 support for Kestrel is still in backlog as it this blocked due to its missing ALPN support in SslStream. You can get some of the HTTP/2 features when using Kestrel (like header compression or multiplexing) if you run it behind a reverse proxy like IIS or NGINX but there is no API to play with. Luckily Kestrel is not the only HTTP server implementation for ASP.NET Core.

HttpSysServer (Formerly WebListener)

The second official server implementation for ASP.NET Core is Microsoft.AspNetCore.Server.WebListener which was renamed to Microsoft.AspNetCore.Server.HttpSys in January. It allows exposing ASP.NET Core applications directly (without a reverse proxy) to the Internet. Under the hood, it's implemented on top of the Windows Http Server API which on one side limits hosting options to Windows only but on the other allows for leveraging the full power of Http.Sys (the same power that runs the IIS). Part of that power is support for HTTP/2 based on which I've decided to build a proof of concept API.

Running an ASP.NET Core Application on HttpSysServer

I've started by creating a simple ASP.NET Core application, something that just runs.

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
    }

    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        app.Run(async (context) =>
        {
            await context.Response.WriteAsync("-- Demo.AspNetCore.Server.HttpSys.Http2 --");
        });
    }
}

Then I've grabbed the source code and compiled it. Now I was able to switch the host to HttpSysServer.

public class Program
{
    public static void Main(string[] args)
    {
        var host = new WebHostBuilder()
            .UseContentRoot(Directory.GetCurrentDirectory())
            .UseStartup()
            .UseHttpSys(options =>
            {
                options.UrlPrefixes.Add("http://localhost:63861");
                options.UrlPrefixes.Add("https://localhost:44365");
                options.Authentication.Schemes = AuthenticationSchemes.None;
                options.Authentication.AllowAnonymous = true;
            })
            .Build();

        host.Run();
    }
}

The two URLs above are kind of a trick from my side - they are the same as ones used by my development instance of IIS Express. The process of configuring SSL for HttpSysServer is a little bit problematic and by using those URLs I've saved myself from going through it as IIS Express has already configured them.

After those steps, I could run the application, navigate to https://localhost:44365 over HTTPS, and see that HTTP/2 has already kicked in (thanks to native support in Http.Sys).

Chrome Developer Tools Network Tab - HttpSysServer responding with H2

HTTP/2 as a Request Feature

The ASP.NET Core has a concept of request features which represent server capabilities related to HTTP. Every request feature is represented by an interface sitting in the Microsoft.AspNetCore.Http.Features namespace. There are features representing web sockets, HTTP upgrades, buffering, etc. Representing HTTP/2 as a feature seems to be in line with this approach.

public interface IHttp2Feature
{
    bool IsHttp2Request { get; }

    void PushPromise(string path);

    void PushPromise(string path, string method, IHeaderDictionary headers);
}

Implementing HTTP/2 With the Windows Http Server API

Deep at the bottom of the HttpSysServer, there is a HttpApi class which exposes the Http Server API. The information on whether the request is being performed over HTTP/2 is available through the Flags field on the HTTP_REQUEST structure. Currently, the field isn't being used so it's simply there as an unsigned integer; and I've decided to change it to flags enum. The second thing that needed to be done was importing the HttpDeclarePush function which allows for Server Push.

internal static unsafe class HttpApi
{
    ...

    [DllImport(HTTPAPI, ExactSpelling = true, CallingConvention = CallingConvention.StdCall,
     CharSet = CharSet.Unicode, SetLastError = true)]
    internal static extern unsafe uint HttpDeclarePush(SafeHandle requestQueueHandle, ulong requestId,
        HTTP_VERB verb, string path, string query, HTTP_REQUEST_HEADERS* headers);

    ...

    [Flags]
    internal enum HTTP_REQUEST_FLAG : uint
    {
        None = 0x0,
        MoreEntityBodyExists = 0x1,
        IpRouted = 0x2,
        HTTP2 = 0x4
    }

    [StructLayout(LayoutKind.Sequential)]
    internal struct HTTP_REQUEST
    {
        internal HTTP_REQUEST_FLAG Flags;
        ...
    }

    ...
}


The IsHttp2Request property should be exposed as part of the request. In order to do that the information needs to be bubbled through two layers. First is NativeRequestContext class, which serves as a bridge to the native implementation and contains a pointer to theHTTP_REQUEST

internal unsafe class NativeRequestContext : IDisposable
{
    ...

    internal bool IsHttp2 => NativeRequest->Flags.HasFlag(HttpApi.HTTP_REQUEST_FLAG.HTTP2);

    ...
}

The second layer is the Request class, which serves as an internal representation of the request. Here we need to grab the value of the NativeRequestContext.IsHttp2 in the constructors because the last step of the constructor is a call to the NativeRequestContext.ReleasePins() which releases the HTTP_REQUEST structure.

internal sealed class Request
{
    internal Request(RequestContext requestContext, NativeRequestContext nativeRequestContext)
    {
        ...

        IsHttp2 = nativeRequestContext.IsHttp2;

        ...

        // Finished directly accessing the HTTP_REQUEST structure.
        _nativeRequestContext.ReleasePins();
    }

    ...

    public bool IsHttp2 { get; }

    ...
}

The Server Push functionality fits better with a response that is internally represented by theResponse class. This is where I'm going to put the method which will take care of transforming the parameters to a form accepted by HttpDeclarePush. The first step is to transform the HTTP method from a string to an HTTP_VERB. Also, some additional validations are needed as only GET and HEAD methods can be used for Server Push.

internal sealed class Response
{
    ...

    internal unsafe void PushPromise(string path, string method, IDictionary<string, StringValues> headers)
    {
        if (Request.IsHttp2)
        {
            HttpApi.HTTP_VERB verb = HttpApi.HTTP_VERB.HttpVerbHEAD;
            string methodToUpper = method.ToUpperInvariant();
            if (HttpApi.HttpVerbs[(int)HttpApi.HTTP_VERB.HttpVerbGET] == methodToUpper)
            {
                verb = HttpApi.HTTP_VERB.HttpVerbGET;
            }
            else if (HttpApi.HttpVerbs[(int)HttpApi.HTTP_VERB.HttpVerbHEAD] != methodToUpper)
            {
                throw new ArgumentException("The push operation only supports GET and HEAD methods.",
                    nameof(method));
            }

            ...
        }
    }
}

The path also needs to be processed, as HttpDeclarePush expects the path portion and query portion separately.

internal sealed class Response
{
    ...

    internal unsafe void PushPromise(string path, string method, IDictionary<string, StringValues> headers)
    {
        if (Request.IsHttp2)
        {
            ...

            string query = null;
            int queryIndex = path.IndexOf('?');
            if (queryIndex >= 0)
            {
                if (queryIndex < path.Length - 1)
                {
                    query = path.Substring(queryIndex + 1);
                }
                path = path.Substring(0, queryIndex);
            }

            ...
        }
    }
}

The hardest part is putting headers into the HTTP_REQUEST_HEADERS structure. The side effect of this process is a list of GCHandle instances which will need to be released after the Server Push (the Response class already contains a FreePinnedHeaders method capable of doing this).

internal sealed class Response
{
    ...

    internal unsafe void PushPromise(string path, string method, IDictionary<string, StringValues> headers)
    {
        if (Request.IsHttp2)
        {
            ...

            HttpApi.HTTP_REQUEST_HEADERS* nativeHeadersPointer = null;
            List<GCHandle> pinnedHeaders = null;
            if ((headers != null) && (headers.Count > 0))
            {
                HttpApi.HTTP_REQUEST_HEADERS nativeHeaders = new HttpApi.HTTP_REQUEST_HEADERS();
                pinnedHeaders = SerializeHeaders(headers, ref nativeHeaders);
                nativeHeadersPointer = &nativeHeaders;
            }

            ...
        }
    }
}

I'm not including the SerializeHeaders method here. If somebody is interested in my certainly not perfect and probably buggy implementation, it can be found here (in general it's based on an already existing SerializeHeaders method which the Response class is using for actual responses).

After all the preparations we can finally call HttpDeclarePush can be called.

internal sealed class Response
{
    ...

    internal unsafe void PushPromise(string path, string method, IDictionary<string, StringValues> headers)
    {
        if (Request.IsHttp2)
        {
            ...

            uint statusCode = ErrorCodes.ERROR_SUCCESS;
            try
            {
                statusCode = HttpApi.HttpDeclarePush(RequestContext.Server.RequestQueue.Handle,
                    RequestContext.Request.RequestId, verb, path, query, nativeHeadersPointer);
            }
            finally
            {
                if (pinnedHeaders != null)
                {
                    FreePinnedHeaders(pinnedHeaders);
                }
            }

            if (statusCode != ErrorCodes.ERROR_SUCCESS)
            {
                throw new HttpSysException((int)statusCode);
            }
        }
    }
}

With the Request and Response classes ready, the feature itself can be implemented. The HttpSysServer aggregates most of the feature's implementations into the FeatureContext class, so this is where the explicit interface implementation will be added.

internal class FeatureContext :
    ...
    IHttp2Feature
{
    ...

    bool IHttp2Feature.IsHttp2Request => Request.IsHttp2;

    void IHttp2Feature.PushPromise(string path)
    {
        ((IHttp2Feature)this).PushPromise(path, "GET", null);
    }

    void IHttp2Feature.PushPromise(string path, string method, IHeaderDictionary headers)
    {
        ...

        try
        {
            Response.PushPromise(path, method, headers);
        }
        catch (Exception ex) when (!(ex is ArgumentException))
        { }
    }

    ...
}

As you can see, I've decided to swallow almost all exceptions coming from Response.PushPromise. This is, in fact, the same approach as in ASP.NET which makes Server Push a fire-and-forget operation (this is ok as the application shouldn't rely on it).

The last step is exposing the new feature as part of the StandardFeatureCollection class. The class provides the _identityFunc field which represents a delegate returning FeatureContext for the current request.


internal sealed class StandardFeatureCollection : IFeatureCollection
{
    ...

    private static readonly Dictionary<Type, Func<FeatureContext, object>> _featureFuncLookup = new Dictionary<Type, Func<FeatureContext, object>>()
    {
        ...
        { typeof(IHttp2Feature), _identityFunc },
        ...
    };

    ...
}

Using the Feature

In order to consume a requested feature, it should be retrieved from the HttpContext.Features collection. If the given feature is not available, the collection will return null. As HttpContext is available on both HttpRequest and HttpResponse classes, the feature can be exposed through some handy extensions.

public static class HttpRequestExtensions
{
    public static bool IsHttp2Request(this HttpRequest request)
    {
        IHttp2Feature http2Feature = request.HttpContext.Features.Get<IHttp2Feature>();

        return (http2Feature != null) && http2Feature.IsHttp2Request;
    }
}
public static class HttpResponseExtensions
{
    public static void PushPromise(this HttpResponse response, string path)
    {
        response.PushPromise(path, "GET", null);
    }

    public static void PushPromise(this HttpResponse response, string path, string method, IHeaderDictionary headers)
    {
        IHttp2Feature http2Feature = response.HttpContext.Features.Get<IHttp2Feature>();

        http2Feature?.PushPromise(path, method, headers);
    }
}

Now it is time to extend the demo application to see this stuff in action. I've created a CSS folder in wwwroot, dropped two simple CSS files in there, and added the StaticFiles middleware. Next, I've modified the code to return some simple HTML referencing the added resources.

public class Startup
{
    ...

    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        app.UseStaticFiles()
            .Map("/server-push", (IApplicationBuilder branchedApp) =>
            {
                branchedApp.Run(async (context) =>
                {
                    bool isHttp2Request = context.Request.IsHttp2Request();

                    context.Response.PushPromise("/css/normalize.css");
                    context.Response.PushPromise("/css/site.css");

                    await System.Threading.Tasks.Task.Delay(100);

                    context.Response.ContentType = "text/html";
                    await context.Response.WriteAsync("<!DOCTYPE html>");
                    await context.Response.WriteAsync("<html>");
                    await context.Response.WriteAsync("<head>");
                    await context.Response.WriteAsync("<title>Demo.AspNetCore.Server.HttpSys.Http2 - Server Push</title>");
                    await context.Response.WriteAsync("<link rel=\"stylesheet\" href=\"/css/normalize.css\" />");
                    await context.Response.WriteAsync("<link rel=\"stylesheet\" href=\"/css/site.css\" />");
                    await context.Response.WriteAsync("</head>");
                    await context.Response.WriteAsync("<body>");

                    await System.Threading.Tasks.Task.Delay(50);
                    await context.Response.WriteAsync($"<h1>Demo.AspNetCore.Server.HttpSys.Http2 (IsHttp2Request: {isHttp2Request})</h1>");
                    await System.Threading.Tasks.Task.Delay(50);

                    await context.Response.WriteAsync("</body>");
                    await context.Response.WriteAsync("</html>");
                });
            })
            .Run(async (context) =>
            {
                await context.Response.WriteAsync("-- Demo.AspNetCore.Server.HttpSys.Http2 --");
            });
    }
}


The delays have been added in order to avoid a client side race between the Server Push and the parser (as the content is really small and response body has higher priority than Server Push, the parser could trigger regular requests for resources instead of claiming pushed ones).

Below is what can be seen in developer tools after running the application and navigating to /server-push over HTTPS.

Chrome Developer Tools Network Tab - HttpSysServer responding with H2

There it is! HTTP/2 with Server Push from ASP.NET Core application.

What's Next

This was a fun challenge. It gave me an opportunity to understand the internals of HttpSysServer and work with a native API which is not something I get to do every day. If somebody would like to roll out their own HttpSysServer with those changes (or have some suggestions and improvements) my full code can be found on GitHub. As there is already an issue for enabling HTTP/2 and server push in the HttpSysServer repository, I'm going to ask the team if this approach is something they would consider a valuable pull request (the IHttp2Feature interface should be probably added to HttpAbstractions, possibly with HttpRequestExtensions and HttpResponseExtensions).

Launch your application faster with Okta’s user management API. Register today for the free forever developer edition!

Topics:
web dev ,asp.net core ,httpsysserver ,http/2

Published at DZone with permission of Tomasz Pęczek. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}