Streamsets Data Collector authentication through LDAP

DZone 's Guide to

Streamsets Data Collector authentication through LDAP

SDC allows user authentication based on files or LDAP. By default, it uses file authentication. This article gives you details on how to switch to use your own company's LDAP.

· Performance Zone ·
Free Resource

StreamSets Data Collector (SDC) is an Open Source lightweight and powerful engine that streams data in real time. It allows to configure data flows as pipelines through a web UI in few minutes. Among its many features, it makes possible to view real-time statistics and inspect data as it passes through the pipeline.

Image title

SDC allows user authentication based on files or LDAP. By default, it uses file authentication. This article gives you details on how to switch to use your own company's LDAP.

To enable LDAP authentication you need to perform the following tasks:

  • Configure the LDAP properties for the Data Collector configuration editing the $SDC_CONF/sdc.properties file:
    • set the value of the http.authentication.login.module property to ldap
  • configure the value of the http.authentication.ldap.role.mapping property to map your LDAP groups to Data Collector roles following this syntax:
    • <LDAP_group>:<SDC_role>,<additional_SDC_role>,<additional_SDC_role>
  • Multiple roles can be mapped to the same group or vice versa. You need to use a semicolon to separate LDAP groups and commas to separate Data Collector roles. Here's an example:
    • http.authentication.ldap.role.mapping=LDAP000:admin;LDAP001:creator,manager;LDAP002:guest
  • The roles you can use are the same (admin, manager, creator, guest) available by default in SDC for the authentication based on files.
  • By default, this property is empty, but it is mandatory to set it when http.authentication.login.module=ldap.
  • Configure the LDAP connection information editing the $SDC_CONF/ldap-login.conf file like in the following example:   
ldap {         
  com.streamsets.datacollector.http.LdapLoginModule required         
  • Where:

    • debug: enables debugging.
    • useLdaps: enables using LDAP over SSL.
    • contextFactory: the initial LDAP context factory. You could leave the default value com.sun.jndi.ldap.LdapCtxFactory
    • hostname: the LDAP server name.
    • port: the LDAP server port.
    • bindDn: the root distinguished name.
    • bindPassword: the connection password. The value can be set here or in a file and then set the reference to that file here.
    • authenticationMethod: the authentication method. You could leave the default value, simple
    • forceBindingLogin: determines if binding login checks are performed. Two possible values for this property. When true, SDC passes the user credentials inputted through the login form to the LDAP server for authentication. When false, SDC performs authentication based on the information received by the LDAP server.
    • userBaseDn: the base distinguished name under which user accounts are located.
    • userRdnAttribute: the name of the username attribute.
    • userIdAttribute: the name of the user ID attribute.
    • userPasswordAttribute: the name of the attribute where the user password is stored.
    • userObjectClass: the name of the user object class.
    • roleBaseDn: the base distinguished name to search for role membership.
    • roleNameAttribute: the name of the attribute for roles.
    • roleMemberAttribute: the name of the role attribute for user names.
    • roleObjectClass: the role object class.
  • In order to check for the proper objects classes, attribute names and values in your company's LDAP, you can use the ldapsearch command-line utility from a Linux machine. This is the syntax of the command in order to retrieve a given user information and the full list of properties for the user object:

ldapsearch -H ldap://<host>:<port> -D "BINDDN" -x -w 'PASSWORD' -b ROLEBASEDN
  • Example:

ldapsearch -H ldap://ldap.googlielmo.org:389 -D "" -x -w 'ldap123' -b "ou=ldap,o=googlielmo.org" "mail=john.smith@googlielmo.org"
  • Finally, don't forget to restart SDC to apply the configuration changes above.

java ,big data ,ldap authentication ,performance ,ruby

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}