DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Creating a Custom Variable Blur Filter in Core Image

Core Image's ease of use and power makes a complicated task simple to implement.

Simon Gladman user avatar by
Simon Gladman
·
Apr. 06, 16 · Tutorial
Like (3)
Save
Tweet
Share
8.46K Views

Join the DZone community and get the full member experience.

Join For Free

Last night, I gave a talk at NSLondon on Apple's Core Image framework. Speaking to attendees afterwards, there were two frequent comments: "Core Image is far easier than I thought" and "Core Image is far more powerful than I thought". 

One of the topics I covered was creating a custom filter to create a variable blur effect - something that may initially sound quite complex but is actually fairly simple to implement. I thought a blog post discussing the custom filter in detail may help spread the word on Core Image's ease of use and awesome power.

This post will discuss the kernel code required to create the effect and the Swift code required to wrap up the kernel in a CIFilter. It's worth noting that Core Image includes its own CIMaskedVariableBlur filter which we'll be emulating in this post.

Core Image Kernels

At the heart of every built in Core Image filter is a kernel function. This is a small program that's executed against every single pixel in the output image and is written in a dialect of OpenGL Shading Language called Core Image Kernel Language. We'll be looking at Core Image's  CIKernel class which manages a kernel function and expects code that looks a little like:

kernel vec4 kernelFunction(sampler image)
{
  return sample(image, samplerCoord(image));
}

In this example, the function accepts a single argument, image, which is of type sampler and contains the pixel data of the source image. The sample function returns the pixel color of the source image at the co-ordinates of the pixel currently being computed and simply returning that value gives is a destination image identical to the source image. 

Variable Blur Kernel

Our filter will be based on a simple box blur. The kernel function will sample neighbouring pixels in a square centred on the pixel currently being computed and return the average color of those pixels. The size of that square is a function of the blur image - the brighter the corresponding pixel in the blur image, the bigger the square.

The code is a little more involved than the previous "pass-through" filter:

kernel vec4 lumaVariableBlur(sampler image, sampler blurImage, float blurRadius) 
{ 
  vec3 blurPixel = sample(blurImage, samplerCoord(blurImage)).rgb; 
  float blurAmount = dot(blurPixel, vec3(0.2126, 0.7152, 0.0722)); 

  int radius = int(blurAmount * blurRadius); 

  vec3 accumulator = vec3(0.0, 0.0, 0.0); 
  float n = 0.0; 

  for (int x = -radius; x <= radius; x++) 
  { 
    for (int y = -radius; y <= radius; y++) 
    { 
      vec2 workingSpaceCoordinate = destCoord() + vec2(x,y); 
      vec2 imageSpaceCoordinate = samplerTransform(image, workingSpaceCoordinate); 
      vec3 color = sample(image, imageSpaceCoordinate).rgb; 
      accumulator += color; +
      n += 1.0; +
    }     
  } 
  accumulator /= n; 

  return vec4(accumulator, 1.0); 
}

In this kernel we:

  • Calculate the blur amount based on the luminosity of the current pixel in the blur image and use that to define the blur radius.
  • Iterate over the surrounding pixels in the input image, sampling and accumulating their values.
  • Return the average of the accumulated pixel values with an alpha (transparency) of 1.0

To use that code in a filter, it needs to be passed in the constructor of a CIKernel:

let maskedVariableBlur = CIKernel(string:"kernel vec4 lumaVariableBlur....")

Implementing as a Core Image Filter

With the CIKernel created, we can wrap up that GLSL code in a Core Image filter. To do this, we'll subclass CIFilter:

class MaskedVariableBlur: CIFilter
{
}

The filter needs three parameters: the input image which will be blurred, the blur image which defines the blur intensity and a blur radius which, as we saw above, controls the size of the sampled area:

var inputImage: CIImage?
var inputBlurImage: CIImage?
var inputBlurRadius: CGFloat = 5

The execution of the kernel is done inside the filter's overridden outputImage getter by invoking applyWithExtenton the Core Image kernel:

override var outputImage: CIImage!
{
  guard let
  inputImage = inputImage,
  inputBlurImage = inputBlurImage else
  {
    return nil
  }

  let extent = inputImage.extent

  let blur = maskedVariableBlur?.applyWithExtent(
    inputImage.extent,
    roiCallback:
    {
      (index, rect) in
      return rect
    },
    arguments: [inputImage, inputBlurImage, inputBlurRadius])

  return blur!.imageByCroppingToRect(extent)
}

The roiCallback is a function that answers the question, "to render the region, rect, in the destination, what region do I need  from the source image?". My book, Core Image for Swift takes a detailed look at how this can affect both performance and the effect of the filter. 

Note that the arguments array mirrors the kernel function's declaration. 

Registering the Filter

To finish creating the filter, it needs to be registered. This is done with CIFilter's static registerFilterNamemethod:

CIFilter.registerFilterName("MaskedVariableBlur",
                            constructor: FilterVendor(),
                            classAttributes: [kCIAttributeFilterName: "MaskedVariableBlur"])

The filter vendor, which conforms to CIFilterConstructor, returns a concrete instance of our filter class based on a string:

class FilterVendor: NSObject, CIFilterConstructor
{
  func filterWithName(name: String) -> CIFilter?
  {
    switch name
    {
      case "MaskedVariableBlur":
        return MaskedVariableBlur()

          default:
        return nil
    }
  }
}

The Filter in Use

Although the filter can accept any image as a blur image, it might be neat to create a radial gradient procedurally (this could even be controlled by a Core Image detector to centre itself on the face!). 

let monaLisa = CIImage(image: UIImage(named: "monalisa.jpg")!)!


let gradientImage = CIFilter(
  name: "CIRadialGradient",
  withInputParameters: [
    kCIInputCenterKey: CIVector(x: 310, y: 390),
    "inputRadius0": 100,
    "inputRadius1": 300,
    "inputColor0": CIColor(red: 0, green: 0, blue: 0),
    "inputColor1": CIColor(red: 1, green: 1, blue: 1)
  ])?
.outputImage?
.imageByCroppingToRect(monaLisa.extent)

Core Image's generator filters, such as this gradient, create images with infinite extent - hence a crop at the end. Our radial gradient looks like:









We can now call our funky new filter using that gradient and an image of the Mona Lisa:

let final = monaLisa
.imageByApplyingFilter("MaskedVariableBlur",
                       withInputParameters: [
                         "inputBlurRadius": 15,
                         "inputBlurImage": gradientImage!])


Which yields this result: where the blur applied to the source is greater where the blur image is lighter:



Core Image for Swift

My book, Core Image for Swift, takes a detailed look at custom filters and covers the high performance warp and color kernels at length. Hopefully, this article has demonstrated that with very little code, even the simplest GLSL can provide some impressive results.

Core Image for Swift is available from both Apple's iBooks Store or, as a PDF, from Gumroad. IMHO, the iBooks version is better, especially as it contains video assets which the PDF version doesn't.

Filter (software) Kernel (operating system)

Published at DZone with permission of Simon Gladman, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • ChatGPT — The Google Killer? The Job Taker? Or Just a Fancy Chatbot?
  • Deploying Java Serverless Functions as AWS Lambda
  • Java Development Trends 2023
  • Too Many Tools? Streamline Your Stack With AIOps

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: