DZone Spotlight

Creating a full-fledged API requires resources, both time and money. You need to think about the model, the design, the REST principles, etc., without writing a single line of code. Most of the time, you don't know whether it's worth it: you'd like to offer a Minimum Viable Product and iterate from there. I want to show how you can achieve it without writing a single line of code. The Solution The main requirement of the solution is to use the PostgreSQL database. It's a well-established Open Source SQL database. Instead of writing our REST API, we use the PostgREST component: PostgREST is a standalone web server that turns your PostgreSQL database directly into a RESTful API. The structural constraints and permissions in the database determine the API endpoints and operations. -- PostgREST Let's apply it to a simple use case. Here's a product table that I want to expose via a CRUD API: Note that you can find the whole source code on GitHub to follow along. PostgREST's Getting Started guide is pretty complete and works out of the box. Yet, I didn't find any ready-made Docker image, so I created my own: Dockerfile FROM debian:bookworm-slim #1 ARG POSTGREST_VERSION=v10.1.1 #2 ARG POSTGREST_FILE=postgrest-$POSTGREST_VERSION-linux-static-x64.tar.xz #2 RUN mkdir postgrest WORKDIR postgrest ADD https://github.com/PostgREST/postgrest/releases/download/$POSTGREST_VERSION/$POSTGREST_FILE \ . #3 RUN apt-get update && \ apt-get install -y libpq-dev xz-utils && \ tar xvf $POSTGREST_FILE && \ rm $POSTGREST_FILE #4 Start from the latest Debian Parameterize the build Get the archive Install dependencies and unarchive The Docker image contains a postgrest executable in the /postgrest folder. We can "deploy" the architecture via Docker Compose: YAML version: "3" services: postgrest: build: ./postgrest #1 volumes: - ./postgrest/product.conf:/etc/product.conf:ro #2 ports: - "3000:3000" entrypoint: ["/postgrest/postgrest"] #3 command: ["/etc/product.conf"] #4 depends_on: - postgres postgres: image: postgres:15-alpine environment: POSTGRES_PASSWORD: "root" volumes: - ./postgres:/docker-entrypoint-initdb.d:ro #5 Build the above Dockerfile Share the configuration file Run the postgrest executable With the configuration file Initialize the schema, the permissions, and the data At this point, we can query the product table: Shell curl localhost:3000/product We immediately get the results: JSON [{"id":1,"name":"Stickers pack","description":"A pack of rad stickers to display on your laptop or wherever you feel like. Show your love for Apache APISIX","price":0.49,"hero":false}, {"id":2,"name":"Lapel pin","description":"With this \"Powered by Apache APISIX\" lapel pin, support your favorite API Gateway and let everybody know about it.","price":1.49,"hero":false}, {"id":3,"name":"Tee-Shirt","description":"The classic geek product! At a conference, at home, at work, this tee-shirt will be your best friend.","price":9.99,"hero":true}] That was a quick win! Improving the Solution Though the solution works, it has a lot of room for improvement. For example, the database user cannot change the data, but everybody can actually access it. It might not be a big issue for product-related data, but what about medical data? The PostgREST documentation is aware of it and explicitly advises using a reverse proxy: PostgREST is a fast way to construct a RESTful API. Its default behavior is great for scaffolding in development. When it’s time to go to production it works great too, as long as you take precautions. PostgREST is a small sharp tool that focuses on performing the API-to-database mapping. We rely on a reverse proxy like Nginx for additional safeguards. -- Hardening PostgREST Instead of Nginx, we would benefit from a full-fledged API Gateway: that enters Apache APISIX. We shall add it to our Docker Compose: YAML version: "3" services: apisix: image: apache/apisix:2.15.0-alpine #1 volumes: - ./apisix/config.yml:/usr/local/apisix/conf/config.yaml:ro ports: - "9080:9080" restart: always depends_on: - etcd - postgrest etcd: image: bitnami/etcd:3.5.2 #2 environment: ETCD_ENABLE_V2: "true" ALLOW_NONE_AUTHENTICATION: "yes" ETCD_ADVERTISE_CLIENT_URLS: "http://0.0.0.0:2397" ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2397" Use Apache APISIX APISIX stores its configuration in etcd We shall first configure Apache APISIX to proxy calls to postgrest: Shell curl http://apisix:9080/apisix/admin/upstreams/1 -H 'X-API-KEY: 123xyz' -X PUT -d ' #1-2 { "type": "roundrobin", "nodes": { "postgrest:3000": 1 #1-3 } }' curl http://apisix:9080/apisix/admin/routes/1 -H 'X-API-KEY: 123xyz' -X PUT -d ' #4 { "uri": "/*", "upstream_id": 1 }' Should be run in one of the Docker nodes, so use the Docker image's name. Alternatively, use localhost but be sure to expose the ports Create a reusable upstream Point to the PostgREST node Create a route to the created upstream We can now query the endpoint via APISIX: Shell curl localhost:9080/product It returns the same result as above. DDoS Protection We haven't added anything, but we're ready to start the work. Let's first protect our API from DDoS attacks. Apache APISIX is designed around a plugin architecture. To protect from DDoS, we shall use a plugin. We can set plugins on a specific route when it's created or on every route; in the latter case, it's a global rule. We want to protect every route by default, so we shall use one. Shell curl http://apisix:9080/apisix/admin/global_rules/1 -H 'X-API-KEY: 123xyz' -X PUT -d ' { "plugins": { "limit-count": { #1 "count": 1, #2 "time_window": 5, #2 "rejected_code": 429 #3 } } }' limit-count limits the number of calls in a time window Limit to 1 call per 5 seconds; it's for demo purposes Return 429 Too Many Requests; the default is 503 Now, if we execute too many requests, Apache APISIX protects the upstream: Shell curl localhost:9080/product HTML <html> <head><title>429 Too Many Requests</title></head> <body> <center><h1>429 Too Many Requests</h1></center> <hr><center>openresty</center> </body> </html> Per-Route Authorization PostgREST also offers an Open API endpoint at the root. We thus have two routes: / for the Open API spec and /product for the products. Suppose we want to disallow unauthorized people to access our data: Regular users can access products, while admin users can access both the Open API spec and products. APISIX offers several authentication methods. We will use the simplest one possible, key-auth. It relies on Consumer abstraction. key-auth requires a specific header: the plugin does a reverse lookup on the value and finds the consumer whose key corresponds. Here's how to create a consumer: Shell curl http://apisix:9080/apisix/admin/consumers -H 'X-API-KEY: 123xyz' -X PUT -d ' #1 { "username": "admin", #2 "plugins": { "key-auth": { "key": "admin" #3 } } }' Create a new consumer Consumer's name Consumer's key value We do the same with consumer user and key user. Now, we can create a dedicated route and configure it so that only requests from admin pass through: Shell curl http://apisix:9080/apisix/admin/routes -H 'X-API-KEY: 123xyz' -X POST -d ' #1 { "uri": "/", "upstream_id": 1, "plugins": { "key-auth": {}, #2 "consumer-restriction": { #2 "whitelist": [ "admin" ] #3 } } }' Create a new route Use the key-auth and consumer-restriction plugins Only admin-authenticated requests can call the route Let's try the following: Shell curl localhost:9080 It doesn't work as we are not authenticated via an API key header. JSON {"message":"Missing API key found in request"} Shell curl -H "apikey: user" localhost:9080 It doesn't work as we are authenticated as user, but the route is not authorized for user but for admin. JSON {"message":"The consumer_name is forbidden."} Shell curl -H "apikey: admin" localhost:9080 This time, it returns the Open API spec as expected. Monitoring A much-undervalued feature of any software system is monitoring. As soon as you deploy any component in production, you must monitor its health. Nowadays, many services are available to monitor. We will use Prometheus as it's Open Source, battle-proven, and widespread. To display the data, we will rely on Grafana for the same reasons. Let's add the components to the Docker Compose file: YAML version: "3" services: prometheus: image: prom/prometheus:v2.40.1 #1 volumes: - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml #2 depends_on: - apisix grafana: image: grafana/grafana:8.5.15 #3 volumes: - ./grafana/provisioning:/etc/grafana/provisioning #4 - ./grafana/dashboards:/var/lib/grafana/dashboards #4 - ./grafana/config/grafana.ini:/etc/grafana/grafana.ini #4-5 ports: - "3001:3001" depends_on: - prometheus Prometheus image Prometheus configuration to scrape Apache APISIX. See the full file here Grafana image Grafana configuration. Most of it comes from the configuration provided by APISIX. Change the default port from 3000 to 3001 to avoid conflict with the PostgREST service Once the monitoring infrastructure is in place, we only need to instruct APISIX to provide the data in a format that Prometheus expects. We can achieve it through configuration and a new global rule: YAML plugin_attr: prometheus: export_addr: ip: "0.0.0.0" #1 port: 9091 #2 Bind to any address Bind to port 9091. Prometheus metrics are available on http://apisix:9091/apisix/prometheus/metrics on the Docker network We can create the global rule: Shell curl http://apisix:9080/apisix/admin/global_rules/2 -H 'X-API-KEY: 123xyz' -X PUT -d ' { "plugins": { "prometheus": {} } }' Send a couple of queries and open the Grafana dashboard. It should look similar to this: Conclusion Creating a full-fledged REST(ful) API is a huge investment. One can quickly test a simple API by exposing one's database in a CRUD API via PostgREST. However, such an architecture is not fit for production usage. To fix it, you need to set a façade in front of PostgREST, a reverse proxy, or even better, an API Gateway. Apache APISIX offers a wide range of features, from authorization to monitoring. With it, you can quickly validate your API requirements at a low cost. The icing on the cake: when you've validated the requirements, you can keep the existing façade and replace PostgREST with your custom-developed API. The source code is available on GitHub. To go further: PostgREST Getting started with Apache APISIX Apache APISIX plugins

In this article, we will discuss testcontainers library and how to use it to simplify our life when it comes to integration testing our code. For the purpose of this example, I am going to use a simple application with its business centered around reviews for some courses. Basically, the app is a service that exposes some GraphQL endpoints for review creation, querying, and deletion from a PostgreSQL database via Spring Data R2DBC. The app is written in Kotlin using Spring Boot 2.7.3. I decided to write this article, especially for Spring Data R2DBC, as with Spring Data JPA, integration testing with testcontainers is straightforward. Still, when it comes to R2DBC, there are some challenges that need to be addressed. Review the App So let’s cut to the chase. Let’s examine our domain. Java @Table("reviews") data class Review( @Id var id: Int? = null, var text: String, var author: String, @Column("created_at") @CreatedDate var createdAt: LocalDateTime? = null, @LastModifiedDate @Column("last_modified_at") var lastModifiedAt: LocalDateTime? = null, @Column("course_id") var courseId: Int ) And here is its repository: Java @Repository interface ReviewRepository : R2dbcRepository<Review, Int> { @Query("select * from reviews r where date(r.created_at) = :date") fun findAllByCreatedAt(date: LocalDate): Flux<Review> fun findAllByAuthor(author: String): Flux<Review> fun findAllByCreatedAtBetween(startDateTime: LocalDateTime, endDateTime: LocalDateTime): Flux<Review> } And here are the connection properties: YAML spring: data: r2dbc: repositories: enabled: true r2dbc: url: r2dbc:postgresql://localhost:5436/reviews-db username: postgres password: 123456 When it comes to testing, Spring offers a fairly easy way to set up an in-memory H2 database for this purpose, but… There is always a but. H2 comes with some disadvantages: First, usually, H2 is not the production DB; in our case, we use PostgreSQL, and it is hard to maintain two DB schemas, one for production use and one for integration testing, especially if you depend on some provider features (queries, functions, constraints and so on). To have as much confidence as possible in tests, it is always advisable to replicate the production environment as much as possible, which is not the case with H2. Another disadvantage might be the possibility of production migration to another database – in this case, H2 having another schema won’t catch any issues, therefore, is unreliable. Others might argue that you can have separate testing dedicated DB on a server, machine, or even in a different schema. Still, again, the hassle of maintaining schemas and the possibility of collision with some other developer`s changes/migrations is way too big. Testcontainers to the Rescue Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container. With Testcontainers, all the disadvantages mentioned above are gone: because you are working with the production-like database, you don’t need to maintain two or more different separate schemas, you don’t need to worry about migration scenarios since you’ll have failing tests, and last, but not least you don’t need to worry about another server/VM’s maintenance since Testcontainers/Docker will take care of that. Here are the dependencies that we are going to use: Groovy testImplementation("org.testcontainers:testcontainers:1.17.3") testImplementation("org.testcontainers:postgresql:1.17.3") There are different ways of working with Testcontainers in Spring Boot. Still, I will show you the `singleton-instance pattern` (one database for all tests) since it is much faster to fire up on a database instance once and let all your tests communicate with it. For this to work, I am going to create an abstract class holding all the Testcontainers instances/start-up creation logic. Java @Tag("integration-test") abstract class AbstractTestcontainersIntegrationTest { companion object { private val postgres: PostgreSQLContainer<*> = PostgreSQLContainer(DockerImageName.parse("postgres:13.3")) .apply { this.withDatabaseName("testDb").withUsername("root").withPassword("123456") } @JvmStatic @DynamicPropertySource fun properties(registry: DynamicPropertyRegistry) { registry.add("spring.r2dbc.url", Companion::r2dbcUrl) registry.add("spring.r2dbc.username", postgres::getUsername) registry.add("spring.r2dbc.password", postgres::getPassword) } fun r2dbcUrl(): String { return "r2dbc:postgresql://${postgres.host}:${postgres.getMappedPort(PostgreSQLContainer.POSTGRESQL_PORT)}/${postgres.databaseName}" } @JvmStatic @BeforeAll internal fun setUp(): Unit { postgres.start() } } } Let’s take a close look at what we have here. Here we make use of testcontainers ability to pull Docker images and therefore instantiate or database container. Java private val postgres: PostgreSQLContainer<*> = PostgreSQLContainer(DockerImageName.parse("postgres:13.3")) Here we make sure to override our R2DBC connection properties in runtime with the ones pointing to our newly created container. Java @JvmStatic @DynamicPropertySource fun properties(registry: DynamicPropertyRegistry) { registry.add("spring.r2dbc.url", Companion::r2dbcUrl) registry.add("spring.r2dbc.username", postgres::getUsername) registry.add("spring.r2dbc.password", postgres::getPassword) } Since we are using Spring Data R2DBC, the spring.r2dbc.url needs a bit more care in order to be properly constructed as, at the moment, PostgreSQLContainer provides only the getJdbcUrl() method. Java fun r2dbcUrl(): String { return "r2dbc:postgresql://${postgres.host}:${postgres.getMappedPort(PostgreSQLContainer.POSTGRESQL_PORT)}/${postgres.databaseName}" } And here, we make sure to start our container before all of our tests. Java @JvmStatic @BeforeAll internal fun setUp(): Unit { postgres.start() } Having this in place, we are ready to write some tests. Java @DataR2dbcTest @Tag("integration") @AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE) class ReviewRepositoryIntegrationTest : AbstractTestcontainersIntegrationTest() { @Autowired lateinit var reviewRepository: ReviewRepository @Test fun findAllByAuthor() { StepVerifier.create(reviewRepository.findAllByAuthor("Anonymous")) .expectNextCount(3) .verifyComplete() } @Test fun findAllByCreatedAt() { StepVerifier.create(reviewRepository.findAllByCreatedAt(LocalDate.parse("2022-11-14"))) .expectNextCount(1) .verifyComplete() } @Test fun findAllByCreatedAtBetween() { StepVerifier.create( reviewRepository.findAllByCreatedAtBetween( LocalDateTime.parse("2022-11-14T00:08:54.266024"), LocalDateTime.parse("2022-11-17T00:08:56.902252") ) ) .expectNextCount(4) .verifyComplete() } } What do we have here: @DataR2dbcTest is a spring boot starter test slice annotation that can be used for an R2DBC test that focuses only on Data R2DBC components. @AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE) - here, we tell spring not to worry about configuring a test database since we are going to do it ourselves. And since we have an R2dbcRepository that delivers data in a reactive way via Flux/Mono, we use StepVerifier to create a verifiable script for our async Publisher sequences by expressing expectations about the events that will happen upon subscription. Cool, right? Let’s run it.io.r2dbc.postgresql.ExceptionFactory$PostgresqlBadGrammarException: [42P01] relation "reviews" does not exist Extensions in Action Bummer! We forgot to take care of our schema and data/records. But how do we do that? In Spring Data JPA, this is nicely taken care of by using the following: Properties files spring.sql.init.mode=always # Spring Boot >=v2.5.0 spring.datasource.initialization-mode=always # Spring Boot <v2.5.0 And by placing a schema.sql with DDLs and data.sql with DMLs in the src/main/resources folder, everything works automagically. Or if you have Flyway/Liquibase, there are other techniques to do it. And even more, Spring Data JPA has @Sql, which allows us to run various .sql files before a test method, which would have been sufficient for our case. But that’s not the case here, we are using Spring Data R2DBC, which at the moment doesn’t support these kinds of features, and we have no migration framework. So, the responsibility falls on our shoulders to write something similar to what Spring Data JPA offers that will be sufficient and customizable enough for easy integration test writing. Let’s try to replicate the @Sql annotation by creating a similar annotation @RunSql Java @Target(AnnotationTarget.FUNCTION) annotation class RunSql(val scripts: Array<String>) Now we need to extend our test’s functionality with the ability to read this annotation and run the provided scripts. How lucky of us that Spring already has something exactly for our case, and it is called BeforeTestExecutionCallback. Let’s read the documentation: BeforeTestExecutionCallback defines the API for Extensions that wish to provide additional behavior to tests immediately before an individual test is executed but after any user-defined setup methods (e.g., @BeforeEach methods) have been executed for that test. That sounds right; let’s extend it and override the beforeTestExecution method. Java class RunSqlExtension : BeforeTestExecutionCallback { override fun beforeTestExecution(extensionContext: ExtensionContext?) { val annotation = extensionContext?.testMethod?.map { it.getAnnotation(RunSql::class.java) }?.orElse(null) annotation?.let { val testInstance = extensionContext.testInstance .orElseThrow { RuntimeException("Test instance not found. ${javaClass.simpleName} is supposed to be used in junit 5 only!") } val connectionFactory = getConnectionFactory(testInstance) if (connectionFactory != null) it.scripts.forEach { script -> Mono.from(connectionFactory.create()) .flatMap<Any> { connection -> ScriptUtils.executeSqlScript(connection, ClassPathResource(script)) }.block() } } } private fun getConnectionFactory(testInstance: Any?): ConnectionFactory? { testInstance?.let { return it.javaClass.superclass.declaredFields .find { it.name.equals("connectionFactory") } .also { it?.isAccessible = true }?.get(it) as ConnectionFactory } return null } } Okay, so what we’ve done here: We take the current test method from the extension context and check for the @RunSql annotation. We take the current test instance – the instance of our running test. We pass the current test instance to getConnectionFactory , so it can take the @Autowired ConnectionFactory from our parent, in our case, the abstract class. AbstractTestcontainersIntegrationTest Using the obtained connectionFactory and ScriptUtils, we execute the scripts found on @RunSql annotation in a blocking-manner. As I mentioned, our AbstractTestcontainersIntegrationTest needs a tiny change; we need to add @Autowired lateinit var connectionFactory: ConnectionFactory so it can be picked by our extension. Having everything in place, it is a matter of using this extension with @ExtendWith(RunSqlExtension::class) and our new annotation @RunSql. Here’s how our test looks now. Java @DataR2dbcTest @Tag("integration") @ExtendWith(RunSqlExtension::class) @AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE) class ReviewRepositoryIntegrationTest : AbstractTestcontainersIntegrationTest() { @Autowired lateinit var reviewRepository: ReviewRepository @Test @RunSql(["schema.sql", "/data/reviews.sql"]) fun findAllByAuthor() { StepVerifier.create(reviewRepository.findAllByAuthor("Anonymous")) .expectNextCount(3) .verifyComplete() } @Test @RunSql(["schema.sql", "/data/reviews.sql"]) fun findAllByCreatedAt() { StepVerifier.create(reviewRepository.findAllByCreatedAt(LocalDate.parse("2022-11-14"))) .expectNextCount(1) .verifyComplete() } @Test @RunSql(["schema.sql", "/data/reviews.sql"]) fun findAllByCreatedAtBetween() { StepVerifier.create( reviewRepository.findAllByCreatedAtBetween( LocalDateTime.parse("2022-11-14T00:08:54.266024"), LocalDateTime.parse("2022-11-17T00:08:56.902252") ) ) .expectNextCount(4) .verifyComplete() } } And here is the content of schema.sql from resources. SQL create table if not exists reviews ( id integer generated by default as identity constraint pk_reviews primary key, text varchar(3000), author varchar(255), created_at timestamp, last_modified_at timestamp, course_id integer ); And here is the content of reviews.sql from resources/data. SQL truncate reviews cascade; INSERT INTO reviews (id, text, author, created_at, last_modified_at, course_id) VALUES (-1, 'Amazing, loved it!', 'Anonymous', '2022-11-14 00:08:54.266024', '2022-11-14 00:08:54.266024', 3); INSERT INTO reviews (id, text, author, created_at, last_modified_at, course_id) VALUES (-2, 'Great, loved it!', 'Anonymous', '2022-11-15 00:08:56.468410', '2022-11-15 00:08:56.468410', 3); INSERT INTO reviews (id, text, author, created_at, last_modified_at, course_id) VALUES (-3, 'Good, loved it!', 'Sponge Bob', '2022-11-16 00:08:56.711163', '2022-11-16 00:08:56.711163', 3); INSERT INTO reviews (id, text, author, created_at, last_modified_at, course_id) VALUES (-4, 'Nice, loved it!', 'Anonymous', '2022-11-17 00:08:56.902252', '2022-11-17 00:08:56.902252', 3); Please pay attention to create table if not exists from schema.sql and truncate reviews cascade from reviews.sql – this is to ensure a clean state of the database for each test. Now, if we run our tests, all is green. We can go ahead and do a little more to follow the DRY principle in regard to the annotations used on this test which can be collapsed under one annotation and then reused, like this. Java @DataR2dbcTest @Tag("integration-test") @Target(AnnotationTarget.CLASS) @ExtendWith(RunSqlExtension::class) @AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE) annotation class RepositoryIntegrationTest() And basically, that’s it, folks. With the common abstract class, test extension, and custom annotation in place, from now on, every new database integration test suite should be a piece of cake. Bonus But wait, why stop here? Having our setup, we can play around with component tests (broad integration tests), for example, to test our endpoints - starting with a simple HTTP call, surfing through all business layers and services all the way to the database layer. Here is an example of how you might wanna do it for GraphQL endpoints. Java @ActiveProfiles("integration-test") @AutoConfigureGraphQlTester @ExtendWith(RunSqlExtension::class) @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) @AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE) internal class ReviewGraphQLControllerComponentTest : AbstractTestcontainersIntegrationTest() { @Autowired private lateinit var graphQlTester: GraphQlTester @Test @RunSql(["schema.sql", "/data/reviews.sql"]) fun getReviewById() { graphQlTester.documentName("getReviewById") .variable("id", -1) .execute() .path("getReviewById") .entity(ReviewResponse::class.java) .isEqualTo(ReviewResponseFixture.of()) } @Test @RunSql(["schema.sql", "/data/reviews.sql"]) fun getAllReviews() { graphQlTester.documentName("getAllReviews") .execute() .path("getAllReviews") .entityList(ReviewResponse::class.java) .hasSize(4) .contains(ReviewResponseFixture.of()) } } And again, if you want to reuse all the annotations there, simply create a new one. Java @ActiveProfiles("integration-test") @AutoConfigureGraphQlTester @ExtendWith(RunSqlExtension::class) @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) @AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE) @Target(AnnotationTarget.CLASS) annotation class ComponentTest() You can find the code on GitHub. Happy coding and writing tests!

Congrats on nailing that interview! You’ve prepared hard and aced all the questions. You can relax now. “Do you have any questions for us?” says the interviewer . Oh! It's not over, you realize. You’re expected to have some questions. Interviewers are not trying to put you on the spot. Neither are they just being polite. The truth is that you are interviewing the company as much as the company is interviewing you. Asking questions at the end of the interview is how you know if they are a good fit for you. Are you interviewing for the job of your dreams or nightmares? Here are a few questions to find out. Questions to Ask the Recruiter The first person you’ll speak with is a recruiter (or someone from human resources), who may not know all the answers, but they can generally get them for you. At this point, the best thing to do is to get a feeling for how the company runs and what to expect next in the interview process. The Position 1. Why is the company hiring? A warming-up question to get a bit of context about the company's state. Is the position open because the company is growing, or are you filling in for someone who left? 2. What happened to the previous person in this position? Did they leave, or were they let go? If you’re filling in for someone who was let go, it may be a good idea to find out what happened. What were the circumstances of the previous employee’s exit? Tactfully learning about past devs can give you insights into what the company expects from you. 3. What is your turnover rate like? How many devs were hired last year, and how many resigned? What's the longest and shortest someone has been on the team? This is where we begin to hunt for flags. A high turnover rate suggests there’s something wrong with the working conditions at the company. You may not get a direct answer. But don't worry. You will have a chance to dig deeper into this issue in later interviews. 4. How does the onboarding process work? What are the next steps in the interview process? Get the information you need to prepare for the next steps. Life as an Employee 5. Does the company invest in employee development, training, or certifications? Is there a budget for learning activities or assisting conferences? It goes without saying that companies are interested in people with a drive to grow. At the very least, a company should give you enough time off to study or attend learning activities. They should get bonus points if they throw in financial aid for education. 6. Does the company allow remote work? How many days am I expected to be at the office? What’s the ratio of remote workers? Do you pay for relocation? The pandemic has pushed workers to remote positions. Some companies are 100% remote, while others offer a mix. What matters the most is if the team you're joining is hybrid or fully remote. With fully remote companies you can also ask if there are regular team building gatherings, hackathons, or celebrations. 7. Is there funding for co-working? Some people find it hard to concentrate when cooped up at home. For them, co-working is a good alternative when the office is too far away. 8. What is the parental leave policy? What is the policy on unpaid leave? What about paid time off like sick leave and vacation days? If these were not communicated in the job description, you should ask what the company’s policy on time off is. Questions to Ask After the Tech Interview At this point, you will probably find yourself talking to someone that can answer technical questions. These kinds of interviews can span various sessions, giving you the chance to talk to future colleagues, tech leads, or the CTO. Take this opportunity to learn about work-life balance and the prevalent workplace culture. A lot of these questions are aimed at exposing red flags. The Daily Loop 9. How many meetings should I expect in a typical week? Meetings are unavoidable, but some companies go overboard. We're trying to assess how much flow time we can expect to properly concentrate. 10. Do you practice CI/CD? What about trunk-based development? Terms like DevOps, Scrum, Lean, and Agile have been abused to the point that they have lost their meanings. Continuous integration (CI), on the other hand, has a more rigorous definition. So the question is: does the company practice it or not? Not practicing CI or trunk-based development is a pretty strong sign that the company relies on manual work to build and test their software. “But Tomas, of course, you would say that. You work for a CI/CD company, after all.” Yeah! But don't take just my word for it. The 2021’s edition of State of DevOps reaffirms the benefits of CI/CD coupled with trunk-based development: “Similar to our findings from previous years, we show that continuous testing is a strong predictor of successful continuous delivery. Elite performers who meet their reliability targets are 3.7 times more likely to leverage continuous testing... Elite performers who meet their reliability targets are 5.8 times more likely to leverage continuous integration... Continuous integration, as defined by Kent Beck and the Extreme Programming community, where it originated, also includes the practice of trunk-based development.” — State of DevOps 2021 Of course, there are places where CI/CD is not feasible. But 99% of the time it is the way to go. 11. How often do you deploy? How do you deploy? We’re starting to dig into the state of the CI/CD pipeline. The thing you want to hear is that they do continuous delivery several times per day, as it implies a fast cycle time. Unless you’re interviewing for a company in a regulated industry, manual and infrequent releases are red flags. They are signs of a slow and unproductive development cycle. 12. Do you practice TDD or BDD? How do you test code? Test-Driven Development and Behavior-Driven Development are disciplines that lead to higher productivity and better designs. Whether you’re in the test-driven camp or not, you should find out how the team tests and designs. One of the gravest red flags out there is a company that does no testing at all. 13. How do you track bugs/issues? What would you say is the ratio between features and fixes? We’re trying to discover the state of the technical debt. Technical debt results from prioritizing new features over fixing or refactoring existing code. Some debt is unavoidable, but accrue too much of it and there's a fair chance you’ll be in the unenviable position of extinguishing fires and triaging messy code. 14. What would you say is more important: don't touch a running system, fixing a bug or working on a new feature? What’s your approach to technical debt? Try asking directly about technical debt, but also see how serious they are about delivering customer value. 15. How well-rounded is the documentation? Do you have coding style guides? Do you have an executable test spec? Try to get a feel of the general state of documentation. Depending on the context, you can ask about API specs, design documents, style guides, user stories, and whatever documents support development. Insufficient documentation often means you’ll have to ask (and later on, be continually asked) for information in order to do your work. Tests such as contract and acceptance tests act as living documentation that validates that the code works according to spec and what has been agreed with the customer. Tools and Documentation 16. What version control system do you use? If the answer is “none,” it’s best to move on and try a different company. Unless you're being interviewed for a team lead or engineering manager position. In which case, ask if they're open to implementing. If they go for it, you're going to have your hands full for a few months at least and it's going to be a difficult road, so factor that into your benefits or salary negotiations. 17. What stacks/languages/frameworks are you using? Don’t worry if you’re not familiar with the technologies. You can learn any stack in a few weeks with good mentoring and dedication. 18. Can I use my ${favorite IDE}? We all have our favorite tools, don't we? 19. Is there company-provided equipment? Will I have root access to my machine? Can I bring my own devices? It might just be me, but I find not having admin access on my work machine disturbing. It shows that the company does not trust its employees. Developer Team Culture 20. Why did you choose to join this company? If you've built a rapport with the interviewer, ask a few personal questions. Knowing the values of the people you’re going to be working for or with is always valuable. 21. How big are the teams? What’s the ratio of juniors vs. seniors? We’re aiming to get insights into the team's composition and size. If you are interviewing for a junior position and the team is composed of a majority of seniors, that's excellent news. There’s nothing more stimulating than being surrounded by people more skilled than you. 22. How many women work for you? What is your process for making sure you have diversity in other ways? It might be a good opening question to discuss the diversity in the team. Tweak the question depending on the context. Refrain from using this question as a form of virtue signaling. A much better strategy is to focus on the facts: evidence shows that a more diverse group gets better solutions. I know diversity can be a charged subject, so try not to make judgments. Always keep the conversation unfailingly polite. 23. What's the biggest mistake you've made at this company? I like this question because it’s deeply related to the concept of generative culture. A generative culture is one where risk is shared, novelty is encouraged, and people are not excoriated for failures (they are used as learning opportunities instead). When people feel psychologically safe, they take more chances and experiment more, leading to innovation. On the opposite side of generative, we have pathological or bureaucratic cultures. In such conditions, people tend to “play it safe” — lest they be punished for failure. It goes without saying that this is not a rewarding environment to work in or one that will likely be good for you, professionally and mentally. Work-Life Balance 24. How many hours do people work in an average week? What time do people typically leave work? The basis of sustainable work without burning out is getting home early enough to have a life. Long work hours can mean the team is unproductive and compensates by overworking. 25. What’s the on-call schedule like? What are standard working hours and your expectations for overtime? How often will I be on call? How often are there emergencies or times when people have to work extra hours? This question complements the previous one. How much overtime happens in a typical month? Lots of overtime, habitual weekend work, and infrequent or manual deployment are signs of an unhealthy work-life balance. If the company shows all these red flags, keep shopping around. 26. Am I expected to be online on Slack/Teams the whole time, or can I batch my work? Much of my productive work is done while away from the keyboard. If you're like me, your best ideas come when walking or taking a shower. Having the chance to step out for a few minutes keeps us in a productive state. Questions to Ask the Manager, CEO, or Founder Once the technical interview torture sessions are over, you will likely get access to a manager, the CEO, or even one of the founders. These are great opportunities to show interest in the company while learning how the company is doing in the market. About the Company 27. Do you have a playbook? The playbook is the single source of truth for all the processes in the company. It describes how the company runs, ensuring that information is accessible to everyone. A playbook allows a new employee to quickly learn the ropes. 28. How do you make money? Are you profitable? How fast are you growing? If you're interviewing for a startup, keep in mind that it will take a few years to be profitable — if they make it at all. So, you’re basically making a high-risk, high-reward bet. Your professional growth will often map to the company’s. If you’re looking for stability and planning to stay for a few years, choosing an established company with a consistent cash flow is better. 29. What are the biggest challenges and opportunities the company or team faces now? Open-ended questions like this can give you a lot of insight into the company's goals and the mindset of management. Check if your professional goals align with those of the company. 30. Where do you see the company in the following 5/10 years? I find it one of the most challenging questions to answer. So, I think it’s only fair to return the favor. 31. How does the company set goals for the quarter/year? And what are they for this period? Shows interest in the company’s goals and, more importantly, will give you an idea of what the priorities will be during this quarter or year. Are they using OKRs (Objectives and Key Results)? What are those for this period? If not, what criteria do they use, and how do they measure their success? If there are no company or team goals, you’re either talking to the wrong person or with the wrong company. About the Position 32. What’s your definition of success for my role? What do you expect me to accomplish in the first 3 months? How will you evaluate my performance at the end of the trial period? How do you determine if someone is a poor fit for your company? It's only fair to know how your performance will be assessed after the trial period. 33. How do performance reviews work? How does the promotion process work? Are performance reviews tied to raises? You and your manager might have different definitions of success. Performance reviews align you, your manager, and the company. Scary as they can be, they facilitate the continuous improvement of people and the company. It is a moment for the manager to give feedback, recognize accomplishments, and provide guidance on career advancement. Without performance reviews, there is no feedback and very little chance of advancement or raises. 34. How much autonomy would I have to decide what to work on? How is work prioritized? Is there any allotted time for side projects/experimentation? We want to know who calls the shots. Take the chance to learn what the team is currently concentrating on. You might be lucky and find a company that allocates a fixed time to work on side projects and experimentation. Advancing your Career 35. Will there be regular 1-on-1s with my manager? You want these. 1-on-1s are enormously important to keep you and the manager on the same page. 36. Can I contribute to FOSS projects? Give talks? Are there any approvals needed? For those interested in contributing to open-source or giving talks while working at the company. These are valuable opportunities for professional growth. Conclusion Some companies will try to show their best side to attract good candidates. Push past the sales pitch and dig deeper. Choose questions that resonate with your interests and keep asking them until you're satisfied. Don't ask about things to which you should already know the answer. Check the job description, the company's website, and all the previous conversations to ensure you don’t sound like a broken record. The list combines questions from several sources (listed below) and my own experience. If you need more inspiration, check the following links: Questions I'm asking in interviews - Julia Evans Questions to ask your future web dev employer 105 Questions to Ask a Company During Your Tech Interview Reverse interview Best of luck, and thanks for reading!

This is an article from DZone's 2022 Performance and Site Reliability Trend Report.For more: Read the Report Software testing is straightforward — every input => known output. However, historically, a great deal of testing has been guesswork. We create user journeys, estimate load and think time, run tests, and compare the current result with the baseline. If we don't spot regressions, the build gets a thumbs up, and we move on. If there is a regression, back it goes. Most times, we already know the output even though it needs to be better defined — less ambiguous with clear boundaries of where a regression falls. Here is where machine learning (ML) systems and predictive analytics enter: to end ambiguity. After tests finish, performance engineers do more than look at the result averages and means; they will look at percentages. For example, 10 percent of the slowest requests are caused by a system bug that creates a condition that always impacts speed. We could manually correlate the properties available in the data; nevertheless, ML will link data properties quicker than you probably would. After determining the conditions that caused the 10 percent of bad requests, performance engineers can build test scenarios to reproduce the behavior. Running the test before and after the fix will assert that it's corrected. Figure 1: Overall confidence in performance metrics Source: Data from TechBeacon Performance With Machine Learning and Data Science Machine learning helps software development evolve, making technology sturdier and better to meet users' needs in different domains and industries. We can expose cause-effect patterns by feeding data from the pipeline and environments into deep learning algorithms. Predictive analytics algorithms, paired with performance engineering methodologies, allow more efficient and faster throughput, offering insight into how end users will use the software in the wild and helping you reduce the probability of defects reaching production. By identifying issues and their causes early, you can make course corrections early in the development lifecycle and prevent an impact on production. You can draw on predictive analytics to improve your application performance in the following ways. Identify root causes. You can focus on other areas needing attention using machine learning techniques to identify root causes for availability or performance problems. Predictive analytics can then analyze each cluster's various features, providing insights into the changes we need to make to reach the ideal performance and avoid bottlenecks. Monitor application health. Performing real-time application monitoring using machine-learning techniques allows organizations to catch and respond to degradation promptly. Most applications rely on multiple services to get the complete application's status; predictive analytics models will correlate and analyze the data when the application is healthy to identify whether incoming data is an outlier. Predict user load. We have relied on peak user traffic to size our infrastructure for the number of users accessing the application in the future. This approach has limitations as it does not consider changes or other unknown factors. Predictive analytics can help indicate the user load and better prepare to handle it, helping teams plan their infrastructure requirements and capacity utilization. Predict outages before it's too late. Predicting application downtime or outages before they happen helps to take preventive action. The predictive analytics model will follow the previous outage breadcrumbs and continue monitoring for similar circumstances to predict future failures. Stop looking at thresholds and start analyzing data. Observability and monitoring generate large amounts of data that can take up to several hundred megabytes a week. Even with modern analytic tools, you must know what you're looking for in advance. This leads to teams not looking directly at the data but instead setting thresholds as triggers for action. Even mature teams look for exceptions rather than diving into their data. To mitigate this, we integrate models with the available data sources. The models will then sift through the data and calculate the thresholds over time. Using this technique, where models are fed and aggregate historical data, provides thresholds based on seasonality rather than set by humans. Algorithm-set thresholds trigger fewer alerts; however, these are far more actionable and valuable. Analyze and correlate across datasets. Your data is mostly time series, making it easier to look at a single variable over time. Many trends come from the interactions of multiple measures. For example, response time may drop only when various transactions are made simultaneously with the same target. For a human, that's almost impossible, but properly trained algorithms will spot these correlations. The Importance of Data in Predictive Analytics "Big data" often refers to datasets that are, well, big, come in at a fast pace, and are highly variable in content. Their analysis requires specialized methods so that we can extract patterns and information from them. Recently, improvements in storage, processors, parallelization of processes, and algorithm design enabled the processing of large quantities of data in a reasonable time, allowing wider use of these methods. And to get meaningful results, you must ensure the data is consistent. For example, each project must use the same ranking system, so if one project uses 1 as critical and another uses 5 — like when people use DEFCON 5 when they mean DEFCON 1 — the values must be normalized before processing. Predictive algorithms are composed of the algorithm and the data it's fed, and software development generates immense amounts of data that, until recently, sat idle, waiting to be deleted. However, predictive analytics algorithms can process those files, for patterns we can't detect, to ask and answer questions based on that data, such as: Are we wasting time testing scenarios that aren't used? How do performance improvements correlate with user happiness? How long will it take to fix a specific defect? These questions and their answers are what predictive analytics is used for — to better understand what is likely to happen. The Algorithms The other main component in predictive analysis is the algorithm; you'll want to select or implement it carefully. Starting simple is vital as models tend to grow in complexity, becoming more sensitive to changes in the input data and distorting predictions. They can solve two categories of problems: classification and regression (see Figure 2). Classification is used to forecast the result of a set by classifying it into categories starting by trying to infer labels from the input data like "down" or "up." Regression is used to forecast the result of a set when the output variable is a set of real values. It will process input data to predict, for example, the amount of memory used, the lines of code written by a developer, etc. The most used prediction models are neural networks, decision trees, and linear and logistic regression. Figure 2: Classification vs. regression Neural Networks Neural networks learn by example and solve problems using historical and present data to forecast future values. Their architecture allows them to identify intricate relations lurking in the data in a way that replicates how our brain detects patterns. They contain many layers that accept data, compute predictions, and provide output as a single prediction. Decision Trees A decision tree is an analytics method that presents the results in a series of if/then choices to forecast specific options' potential risks and benefits. It can solve all classification problems and answer complex issues. As shown in Figure 3, decision trees resemble an upsidedown tree produced by algorithms identifying various ways of splitting data into branch-like segments that illustrate a future decision and help to identify the decision path. One branch in the tree might be users who abandoned the cart if it took more than three seconds to load. Below that one, another branch might indicate whether they identify as female. A "yes" answer would raise the risk as analytics show that females are more prone to impulse buys, and the delay creates a pause for pondering. Figure 3: Decision tree example Linear and Logistic Regression Regression is one of the most popular statistical methods. It is crucial when estimating numerical numbers, such as how many resources per service we will need to add during Black Friday. Many regression algorithms are designed to estimate the relationship among variables, finding key patterns in big and mixed datasets and how they relate. It ranges from simple linear regression models, which calculate a straight-line function that fits the data, to logistic regression, which calculates a curve (Figure 4). OVERVIEW OF LINEAR AND LOGISTIC REGRESSION Linear Regression Logistic Regression Used to define a value on a continuous range, such as the risk of user traffic peaks in the following months. It's a statistical method where the parameters are predicted based on older sets. It best suits binary classification: datasets where y = 0 or 1, where 1 represents the default class. Its name derives from its transformation function being a logistic function. It's expressed as y = a + bx, where x is an input set used to determine the output y. Coefficients a and b are used to quantify the relation between x and y, where a is the intercept and b is the slope of the line. It's expressed by the logistic function:where β0 is the intercept and β1 is the rate. It uses training data to calculate the coefficients, minimizing the error between the predicted and actual outcomes. The goal is to fit a line nearest to most points, reducing the distance or error between y and the line. It forms an S-shaped curve where a threshold is applied to transform the probability into a binary classification. Figure 4: Linear regression vs. logistic regression These are supervised learning methods, as the algorithm solves for a specific property. Unsupervised learning is used when you don't have a particular outcome in mind but want to identify possible patterns or trends. In this case, the model will analyze as many combinations of features as possible to find correlations from which humans can act. Figure 5: Supervised vs. unsupervised learning Shifting Left in Performance Engineering Using the previous algorithms to gauge consumer sentiment on products and applications makes performance engineering more consumer-centric. After all the information is collected, it must be stored and analyzed through appropriate tools and algorithms. This data can include error logs, test cases, test results, production incidents, application log files, project documentation, event logs, tracing, and more. We can then apply it to the data to get various insights to: Analyze defects in environments Estimate the impact on customer experience Identify issue patterns Create more accurate test scenarios, and much more This technique supports the shift-left approach in quality, allowing you to predict how long it will take to do performance testing, how many defects you are likely to identify, and how many defects might make it to production, achieving better coverage from performance tests and creating realistic user journeys. Issues such as usability, compatibility, performance, and security are prevented and corrected without impacting users. Here are some examples of information that will improve quality: Type of defect In what phase was the defect identified What the root cause of the defect is Whether the defect is reproducible Once you understand this, you can make changes and create tests to prevent similar issues sooner. Conclusion Software engineers have made hundreds and thousands of assumptions since the dawn of programming. But digital users are now more aware and have a lower tolerance for bugs and failures. Businesses are also competing to deliver a more engaging and flawless user experience through tailored services and complex software that is becoming more difficult to test. Today, everything needs to work seamlessly and support all popular browsers, mobile devices, and apps. A crash of even a few minutes can cause a loss of thousands or millions of dollars. To prevent issues, teams must incorporate observability solutions and user experience throughout the software lifecycle. Managing the quality and performance of complex systems requires more than simply executing test cases and running load tests. Trends help you tell if a situation is under control, getting better, or worsening — and how fast it improves or worsens. Machine learning techniques can help predict performance problems, allowing teams to course correct. To quote Benjamin Franklin, "An ounce of prevention is worth a pound of cure." This is an article from DZone's 2022 Performance and Site Reliability Trend Report.For more: Read the Report

Non-fungible tokens (NFTs) are one of the biggest (and most popular) innovations to come out of Web3 so far. Although their capabilities are still being explored, both the art industry and the gaming world have benefited from the technology. This is because NFTs create an indisputable record of digital ownership. Building NFTs isn’t as complicated as you think, especially with tool suites such as Truffle Suite. This article will introduce you to the Truffle Suite of tools and teach you how to use them to build an end-to-end NFT project and include the following: Project setup and installation using the Truffle NFT Box Introduction to the Truffle VS Code extension How to build and deploy the NFT smart contract using the Truffle Dashboard Instructions for minting an NFT from the built-in React client with the Truffle NFT Box Verification on the block explorer and OpenSea/Rarible Introduction to the Truffle Suite To make life easier for Ethereum developers, Truffle provides a development environment, testing framework, and asset pipeline. It has several features that make it appealing to both experienced and new-to-Web3 devs, and it enables developers to build and deploy dapps for many use cases. The Truffle Suite of tools has several components, including: A programmable build pipeline used to build console and web applications Instant asset rebuilding while in development (truffle watch) A console that makes it simple to use your compiled contracts (truffle console) Contract compilation and deployment with built-in support for JavaScript, CoffeeScript, ES6, and JSX utilizing the RPC client of your choice Truffle Suite Truffle comprises essential tools to help anyone quickly build and deploy Web3 projects. The tools in the stack include: Truffle - The development environment and testing framework Ganache - A personal blockchain for Ethereum development Truffle For VS Code Extension - An extension that makes creating and managing Truffle projects easier Truffle Dashboard - An easy way to use your existing MetaMask wallet for your deployments and other transactions you need to send from a command line context Truffle Boxes - Boilerplate projects to get up and running quickly Although not part of the Truffle Suite of tools, the following tools pair nicely with the stack to make development and deployment even more streamlined: Infura - A Web3 backend and Infrastructure-as-a-Service (IaaS) provider, offering various services and tools to blockchain developers MetaMask - A crypto wallet and gateway to blockchain applications How To Launch an End-to-End NFT Project Using the Truffle Stack Now that you’ve got an idea of the tools involved, let’s put them together to create an NFT project. Prerequisites Before getting started, we need the following prerequisites: Node.js and its package manager NPM Verify we have Node.js installed by using the following terminal command: node -v && npm -v An Infura account VS Code Extension for Truffle Basic understanding of JavaScript and Solidity Create an Infura Account to Access the Network Visit the Infura website to sign up for a new account or log in if you already have one. After successfully signing up, the page redirects to our dashboard, where we can create a new key, as shown below. Click the "Create a New Key" button and fill in the required information. After creating your key, your project ID will be visible on your dashboard under the API Key section, as shown below. Copy and keep it somewhere; you will need it later in this tutorial. Set Up MetaMask with Infura RPC We will need to configure and set up a Goerli test network on MetaMask using the Infura RPC endpoint since you will build and deploy your application via the Truffle Dashboard. Head over to metamask.io and download the extension for your browser. Understanding wallet safety is essential when using a crypto wallet like MetaMask. Click on Add a Network Manually as shown in the image above, then enter the following information: Network Name: Goerli Testnet - Infura New RPC URL: https://goerli.infura.io/v3/<Paste-your-project-id-here> Chain ID: 5 Currency Symbol: ETH Block Explorer URL: https://goerli.etherscan.io/ Congratulations, you have successfully configured the Goerli Testnet with Infura RPC. End-to-End NFT Project Setup with the Truffle NFT Box and Truffle for VS Code Extension In this section, you will set up an NFT project with the Truffle NFT Box and the Truffle for VS Code extension. Open VS Code from the folder you wish to work out of, navigate to the extensions tab, and search “Truffle for VS Code.” With the extension installed, you can easily create a new Truffle project from a Truffle Box. Press ctrl+shift+P to open the command dialogue and type “truffle.” This will display a list of available Truffle commands. Select New Solidity Project and then Create project from Truffle Box. Next, a list of available Truffle Boxes will be displayed. Choose nft-box to create the project. Alternatively, if you are not using VS Code as your main editor, you can still follow along. Create an NFT project from a Truffle Box in any directory of your choice using the following commands: mkdir nft-project-demo && cd nft-project-demo npx truffle unbox nft-box Or you can install Truffle globally and run the unbox command. npm install -g truffle truffle unbox nft-box This command should download and unbox the Truffle NFT Box. Next, cd into the new directory and download all project dependencies by running npm install. Finally, open the project in your preferred code editor. Regardless of how you created the project, you should have a project structure similar to what is shown below. Upload NFT Metadata on IPFS Our NFT image and metadata will be stored using IPFS. Head over to your Infura Dashboard and create another project, this time selecting IPFS as shown below. Next, click Manage Key to view the project details. You'll need the Project ID, Project Secret, and Endpoint URL for the following tasks. Using the following command, let's upload the image to IPFS. To upload the image, cd into the folder where the image is saved. curl -X POST -F file=@yourfile -u "PROJECT_ID:PROJECT_SECRET" "https://ipfs.infura.io:5001/api/v0/add" Replace @yourfile with the name of the image file in the current folder, e.g., image-name.jpeg. Replace "PROJECT_ID:PROJECT_SECRET" with your project id and secret in that order. The output from the command should look similar to what is shown below. You can verify the image was uploaded successfully by pasting that Hash into the following URL: https://ipfs.io/ipfs/YOUR_HASH_HERE. Note: When trying to access the image via the URL for the first time, it might take a few minutes to load. Let's get the metadata stored on IPFS now that the image is there. To comply with OpenSea requirements, you will need your metadata to be in the following JSON format: { "name": "My Nature NFT", "description": "Amazing nature NFT deployed to the Goerli Network using Infura RPC", "image": "ipfs://YOUR_HASH_HERE" } You will use the command similar to what you used earlier to upload the image in the previous step with this command. curl -X POST -F file=@yourjsonfile -u "PROJECT_ID:PROJECT_SECRET" "https://ipfs.infura.io:5001/api/v0/add" Replace @yourjsonfile with the name of the json file in the current folder, e.g., nft-metadata.json. Replace "PROJECT_ID:PROJECT_SECRET" with your project id and secret in that order. You can verify the JSON file was uploaded successfully by pasting that Hash into the following URL: https://ipfs.io/ipfs/YOUR_HASH_HERE You have entirely set up all the tools we need to build and deploy a smart contract with the Truffle Dashboard. Let's do that in the following step. Build and Deploy Smart Contracts with the Truffle Dashboard Head over to the NFT project with the Truffle NFT Box we installed earlier; you will build and deploy the smart contract in this section. Inside the 1_deploy_contracts.js file in the migrations folder, update it with the following code snippet. var NFTCollection = artifacts.require("./NFTCollection.sol"); module.exports = function (deployer) { deployer.deploy( NFTCollection, "https://ipfs.io/ipfs/YOUR_IPFS_HASH_HERE", "NFT DEMO", "NFTD" ); }; In the code snippet above, we added the URL to the hosted JSON file on IPFS, the token name, and the symbol as the parameter to the deployer.deploy() function. Next, you will build and compile using the Truffle Dashboard, which removes the need to interact manually with your wallet's mnemonic phrase or private keys throughout the development lifecycle. In a separate terminal window, type the following command: truffle dashboard The command will automatically start Truffle Dashboard at http://localhost:24012 and open a new tab in your existing browser session. After connecting your wallet, make sure you switch to the Goerli testnet we set up and configured earlier using the Infura RPC. You should have something similar to what is shown below after successfully connecting. Next, compile and deploy the smart contracts using the following command: truffle migrate --network dashboard Approve the transaction via the Truffle Dashboard on your browser as prompted on the terminal. After successful deployment, you should have something similar to what is shown below on your terminal. Verify Contract Creation on Goerli Testnet Block Explorer In this section, verify the contract creation on the Goerli Testnet Block Explorer via the testnet URL by pasting the contract address or confirming from MetaMask, which will redirect to the Goerli Testnet Block Explorer as shown below. Configure the Frontend Client and Mint an NFT Open a separate terminal and run the following command to configure the front-end client. This will allow us to mint the NFT from the smart contract we just deployed. cd client npm install npm start If you encounter an error similar to what is shown above, create a .env file in the client folder, add SKIP_PREFLIGHT_CHECK=true, then run the following command. npm start The React server will automatically start on the browser at http://localhost:3000/, showing something similar to the image below: Test the End-to-End NFT Project In this section, let's test the application to see if you can successfully mint the NFT by clicking on the Mint NFT button. After pressing the button, you will see a MetaMask pop-up similar to what is shown below. By clicking on the transaction below, you can confirm whether the minting is successful on MetaMask. Verify Minted NFT on OpenSea and Rarible We successfully built and deployed an end-to-end NFT project using the Truffle stack in the previous steps. In this section, let's verify our minted NFT on OpenSea and Rarible. Copy the contract address as shown below. Paste your contract address into the search bar at testnets.opensea.io to check out your new NFT on OpenSea. If your metadata was entered correctly, your NFT should display properly. Note: OpenSea can sometimes be slow to show your NFT metadata and image. You can paste your contract address and check it out on Rarible, as they retrieve NFT metadata faster than OpenSea, as shown below. As you can see that our NFT properly displays on both OpenSea and Rarible testnet platforms, thus verifying a successful deployment. That's it! You can find the complete code on this GitHub repository. Conclusion This article introduced you to the Truffle Suite of tools and how they can be used with Infura and MetaMask to create an end-to-end NFT project. You also learned that Truffle Boxes are an easy way to build projects quickly. Development in Web3 does not need to be difficult, especially when there are tools you can use in every step of the process. To learn more about the Truffle stack or to continue building, check out their docs. Have a really great day!

Horizontal scaling, and, in particular, auto-scaling is a common challenge for many applications no matter their architectural approach (monolithic or microservices) and the deployment environment (cloud or on-premises). Today's cloud providers provide a rich variety of options and tools to accommodate this critical requirement, which is to be able to scale services instances up or down, based on specified rules, in an efficient manner in order to: Avoid unavailability Ensure the quality of service Optimize resources utilization Avoid unnecessary charges Many of the out-of-the-box solutions rely on Kubernetes and offer various levels of abstraction to make this process as smooth as possible. However, there are many cases where we don't really need or cannot afford all these cloud offerings as-a-service or we need to operate our own data center on-premises. As presented in a previous article "Look, Ma! No Pods!", HashiCorp Nomad is a simple and flexible deployment tool, able to manage both containers and non-containerized applications running on-prem, cloud, or hybrid environments. Lately, Nomad provides auto-scaling functionality via the Nomad Autoscaler Agent. As described in the Nomad Autoscaler Overview, the following strategies are supported: Horizontal Application Autoscaling: This is the process of automatically controlling the number of instances of an application to have sufficient work throughput to meet service-level agreements (SLA). Horizontal Cluster Autoscaling: The process of adding or removing Nomad clients from a cluster to ensure there is an appropriate amount of cluster resource for the scheduled applications Dynamic Application Sizing: Enterprise feature which enables organizations to optimize the resource consumption of applications using sizing recommendations from Nomad In this article, we will delve into a Horizontal Application Autoscaling use-case. More specifically we will: Spin up a minimal Nomad/Consul cluster on Google Cloud's Compute Engine VMs, using HashiCorp's IaC tool Terraform. Deploy via Nomad the following components: the Autoscaler Agent, an HAProxy Load Balancer, a Prometheus server, a Spring Native application, and a Redis standalone server which is used by the application as an in-memory data store. Generate some load against the Spring application in order to trigger the auto-scaling process and observe the overall system's behavior. Pre-Requisites In order to be able to run everything you need to: Create a new Google Cloud account or sign in to an existing one Create a new Google Cloud project (guide here) Download and install Google Cloud Command Line Interface (gcloud CLI) to your development machine Download and install HashiCorp Packer Download and install Terraform You then need to clone the following GitHub repositories: Terraform Google Nomad: contains the Terraform scripts used to provision the Nomad/Consul cluster on GCP. This is actually a fork from HashiCorp* with the following tweaks applied: Updated Ubuntu, Nomad, and Consul versions Additions to install Docker daemon on each provisioned VM Nomad configuration changes for enabling Telemetry and exporting metrics to Prometheus Minimal changes to Terraform .tf files to avoid errors with the latest versions of Terraform CLI Spring Boot Yaus contains the source code of the sample Spring Boot native app used for the demo, a Nomad job file for its deployment, and a JMeter Test Plan for the load testing *Side-note: The particular fork is selected due to familiarity with it at the time of writing. You are encouraged to also check the more recent nomad/terraform GitHub repo especially if you need to run the Nomad/Consul cluster localhost or use another Cloud Provider. Another alternative is to use the Caravan Open Source tool. Keep in mind that if you are interested in running the cluster on a local Windows machine using Vagrant, you need to disable the Hyper-V capability and this may cause side effects to your Docker Desktop. Spin Up a Nomad/Consul Co-Located Cluster Build a Google Image Open your gcloud CLI and authenticate to your account by executing: gcloud auth application-default login From within gcloud CLI, change the directory to the `terraform-google-nomad/examples/nomad-consul-image` folder. Then execute: packer build nomad-consul.json This will take some time and when the build finishes, it will output the name of a produced new Google Image, e.g., "nomad-consul-ubuntu18-2022-11-11-110511". This is actually an Ubuntu image, with Nomad & Consul client/server binaries plus Docker installed. Keep the image name at hand for the next step. Customize Terraform Input Variables From within gcloud CLI, go back to the `terraform-google-nomad` root directory and edit the variables.tf file in order to specify: The name of the GCP Project where all resources will be launched The GCP Region and Zone you wish the VMs to be provisioned The size of the Cluster (number of servers and clients) The machine type of the Compute Instance to run for each node type The namespace of the Nomad/Consul Server cluster The namespace of the Nomad client cluster The Google Image used to launch each node in the Nomad/Consul Server cluster The Google Image used to launch each node in the Nomad client cluster Use the same Google Image identifier for both nomad_consul_server_source_image and nomad_client_source_image. Based on the existing setup, Terraform configuration will take care of executing Nomad and Consul on either client or server mode automatically. Once you finish, save and exit the editor. Plan and Deploy the Infrastructure Now, from the same, root folder of the cloned project you may execute: terraform init And then you may run: terraform plan If the plan looks good, run: terraform apply Once the procedure finishes, you can have a look at the VM instances provisioned from your Google Cloud Console. It should look like the one below: Note that with the External IP of the nomad-server-* machine, you should be able to access Nomad and Consul admin UIs at HTTP://<nomad-server-external-ip>:4646 and HTTP://<nomad-server-external-ip>:8500 respectively. Terraform has taken care of all the internal and external network and firewall plumbing in order for the cluster to be functional and easily accessible, solely for testing purposes of course. Deploy Servers and Applications Now that we have our Nomad and Consul cluster up and running we can proceed with deploying Nomad Autoscaler Agent, Prometheus, HAProxy, and the sample microservice along with the Redis server it needs. Navigate to your Nomad admin UI, select "Run Job" and paste in the Job Definition area, the contents of the autoscaler.nomad job. Select "Plan" and if it looks OK, press "Run". Repeat the process for haproxy.nomad and prometheus.nomad. Notice that in our Nomad Job files we make use of the so-called template Stanza. This is a powerful method of generating configuration files on-the-fly, which are populated from environment variables, Consul Service discovery metadata, HashiCorp Vault secrets, etc. Let's see how this is useful, particularly with HAProxy's configuration. Check out the following excerpt from haproxy.nomad file: Plain Text backend http_back balance roundrobin server-template mywebapp 10 _demo-webapp._tcp.service.consul resolvers consul resolve-opts allow-dup-ip resolve-prefer ipv4 check With the above configuration, HAProxy will be able to discover dynamically via the Consul Service Registry, the instances (IPs and ports) of the application it will load balance. No need for a static or manually updated configuration, everything is referred to by service name! Cool, right? You may finish up the deployments by running redis.nomad and demo-webapp.nomad. By now, theoretically, you should have everything up and running, so we have an overview from the Nomad UI Jobs listing view: And the Consul UI Services listing view: Nomad Autoscaler Basics The following diagram depicts the Nomad Autoscaler architecture as initially described in HashiCorp Nomad Autoscaling Tech Preview, focusing on horizontal application scaling: For auto-scaling applications, Nomad provides a plugin interface through which we can connect multiple data stores for metrics. For example, if using Prometheus for monitoring, we can easily integrate it with the Prometheus plugin and define the scaling policies using the PromQL expressions. Let's have a look at the scaling Stanza, from the Spring microservice Nomad Job: JSON scaling { enabled = true min = 1 max = 4 policy { evaluation_interval = "2s" cooldown = "5s" check "cpu_usage" { source = "prometheus" query = "avg(nomad_client_allocs_cpu_total_percent{task='server'})" strategy "target-value" { target = 25 } } } } In simple words, we declare that we want Nomad to scale up to 4 instances of our app, if the average CPU load of the nodes running this task reaches 25%. There are many alternatives to try out, as long as there is a metric you are interested in, existing in Prometheus! Generate Load and Observe Time to generate some traffic using a sample JMeter Test Plan provided here. You need to open it with JMeter and modify the Server IP Address in the "HTTP Request" section: The Sample Spring Boot Native App The sample app is a simplified "URL Shortener" and exposes 2 endpoints that can be accessed via the HA_Proxy's External IP and port. The particular app is a Spring Boot native application built using Cloud Native Buildpacks in a Docker image. Spring Native provides support for compiling Spring applications to native executables using the GraalVM native-image compiler. Compared to the Java Virtual Machine, using native images provides key advantages, such as instant startup, instant peak performance, and reduced memory consumption. So, as far as auto-scaling is concerned we can surely benefit from all these advantages! The complete source code plus Nomad job files are available at the Spring Boot native app demo on GitHub (linked earlier in this post). Prometheus UI Once the traffic starts to spike and the CPU load increases, you can observe and confirm it from Prometheus UI: Nomad and Consul Observability At the same time, from the Nomad UI you can also tail the Nomad Autoscaler Agents LOGs: Confirm that Nomad begins to spin up more instances of the app, respecting our scaling policy. Look for lines like target=nomad-target from=1 to=2 reason="scaling up because the factor is XXX", target=nomad-target from=2 to=1 reason="scaling down because the factor is YYY", and possible errors or warnings that may be a sign that you need to apply a more sophisticated tuning. The additional instances are discovered dynamically by the HAProxy load balancer and start to take their share of the load. The number of service instances, their IPs, and ports are also visible in the Consul admin UI. Tearing It Down Once you stop the JMeter load test, eventually the application will scale down and remain with a single instance running. Finally, you may proceed to "destroy" the ephemeral GCP testbed, by executing from within your gcloud CLI the command: terraform destroy Closing Thoughts Horizontal scaling is a key characteristic of almost every solution; therefore, its architecture must be able to support it if and when needed. Auto-scaling is not always mandatory, especially if we are dealing with very predictable workloads. In this guide, we presented HashiCorp Nomad's way with a Spring Boot native microservice as the scaling target.

What Is Kubernetes Lens? Lens is an open-source integrated development environment (IDE) that allows users to connect and manage multiple Kubernetes clusters on Mac, Windows, and Linux platforms. It provides a feature-rich, intuitive graphical interface that allows users to deploy and manage clusters directly from the console. Kubernetes Lens provides built-in dashboards that provide key metrics and insights into resources running on a Kubernetes cluster, including deployment, configuration, networking, storage, access control, and custom resources. Lens was originally developed by Finnish tech startup Kontena. In 2020, it was acquired by Mirantis and released as an open-source project under the MIT license. Kubernetes Lens Alternatives Kubernetes Dashboard Kubernetes Dashboard on GitHub License: Apache License 2.0 The Kubernetes Dashboard, shipped with the core distribution of Kubernetes, is one of the most popular and mature GUI clients for Kubernetes. It is a web-based UI that provides an overview of workloads running on a cluster, and lets users create or modify individual Kubernetes resources. Its visualization and filtering capabilities are limited compared to other solutions like Lens and Octant, and it does not support organizing resources by tags. To use the Kubernetes Dashboard, users need to install it in a Kubernetes cluster and handle user login and access rights issues. Setting up the relevant authentication for the web host UI is complex — the default setup requires dashboard users to supply a token or upload a KubeConfig file. Istio Istio GitHub License: Apache License 2.0 Istio is an independent, open-source service mesh technology that allows developers to connect, secure, control, monitor, and run distributed microservices architectures (MSAs) regardless of platform, origin, or vendor. Istio helps developers overcome the loss of observability and interaction control that comes with a growing number of microservices. It can also be instrumental in securing Kubernetes and preventing unauthorized access to microservices. Istio manages service interactions for container and virtual machine (VM)-based workloads. Together with Kiali, an open source visualization tool, Istio can be used to visualize Kubernetes cluster workloads, interactions between them, and their network communication. Octant Octant GitHub License: Apache License 2.0 Octant is a tool that helps developers to understand how their applications run on Kubernetes clusters. It provides a combination of self-examination tools, cluster navigation, and object management, together with a plug-in system to further extend its capabilities. Key features include: Resource viewer — graphically visualizes the relationships between objects in a Kubernetes cluster. The health of individual objects is colored to indicate workload performance. View overview — a single page with consolidated status and configuration information, aggregated from the data found in the outputs of kubectl commands. Port forward — allows debugging applications by forwarding local ports to running pods and forwarding multiple pods between namespaces. Log stream — displays log streams from pods and containers for troubleshooting and monitoring, without opening multiple terminals. Label filter — configures a workload with label filtering to inspect clusters that have many objects in one namespace. Cluster exploration — ability to change namespaces or contexts between different clusters, with support for multiple kubeconfig files. Plugin system — an extensible plugin system that allows users to provide additional functionality via gRPC. Plugin authors can add components on top of existing views. Rancher Dashboard Rancher Dashboard on GitHub License: Apache License 2.0 Rancher is a popular Kubernetes multi-cluster management solution, which includes the Rancher Dashboard, a stateless client for the Rancher API built with Vue.js and Nuxt. It is built and packaged as a folder of static HTML/CSS/JS files bundled with the Rancher release. The Rancher Dashboard shows all Kubernetes object types, namespaces and operations that the logged in user has access to. All default views are raw YAML from the Kubernetes API, which can also be organized in tabular form for list pages. It lets users graphically edit resources instead of editing YAML, by customizing table columns and their format. Skooner Skooner on GitHub License: Apache License 2.0 Skooner is a Kubernetes dashboard with configuration views, workload views, management and YAML editing directly in the UI. Key features include: Cluster management — including namespaces, nodes, pods, replica sets, deployments, storage, and RBAC. Fast updates — shows up-to-date cluster status without refreshing the page. Visualize cluster health — real-time graphs let users quickly identify degraded resources. Easy CRUD and extensions — inline API documentation allows users to easily understand what each field does. Responsive design — runs seamlessly on mobile phones and tablets.. OpenID integration with no special proxy required. Easy installation — it is possible to deploy Skooner in less than a minute with a few YAML resources. Conclusion In this article, I introduced Kubernetes Lens and reviewed 5 great tools you should consider as alternatives: Kubernetes Dashboard — the classic dashboard included with the core K8s distribution Istio — a popular service mesh solution that enables visualization when combined with Kiali Octant — a full-featured observability and troubleshooting solution Rancher Dashboard — the built-in dashboard in the open source Rancher Kubernetes distribution Skooner — a lightweight Kubernetes dashboard solution I hope this will be useful as you level up your Kubernetes observability toolset.

This is an article from DZone's 2022 Performance and Site Reliability Trend Report.For more: Read the Report In the past few years, the complexity of systems architectures drastically increased, especially in distributed, microservices-based architectures. It is extremely hard and, in most cases, inefficient to debug and watch logs, particularly when we have hundreds or even thousands of microservices or modules. In this article, I will describe what observability and monitoring systems, the patterns of a good observability platform, and the observability subsystem may look like. Observability vs. Monitoring Before we jump directly to the point, let's describe what observability is, what components it includes, and how it differs from monitoring. Observability allows us to have a clear overview of what happens in the system without knowing the details or domain model. Moreover, observability lets us efficiently provide information about: The overall system, separate service failures, and outages The behavior of the general system and services The overall security and alerts We know what functions should cover the observability system. Below we can see what information should be gathered to properly design an observability and monitoring platform. Metrics – Data collection allows us to understand the application and infrastructure states — for example, latency and the usage of CPU, memory, and storage. Distributed traces – Allows us to investigate the event or issue flow from one service to another Logs – This is a message with a timestamp that contains information about application- or service-level errors, exceptions, and information. Alerting – When an outage occurs or something goes wrong with one or several services, alerts notify these problems via emails, SMS, chats, or calls to operators. This allows for quick action to fix the issue. Availability – Ensures that all services are up and running. The monitoring platform sends prob messages to some service or component (to the HTTP API endpoint) to check if it responds. If not, then the observability system generates an alert (see the bullet point for alerting). Also, some observability and monitoring platforms may include user experience monitoring, such as heat maps and user action recording. Observability and monitoring follow the same principles and patterns and rely primarily on toolsets, so in my opinion, the differentiation between the two is made for marketing purposes. There is no clear definition of how observability differs from monitoring; all definitions are different and high-level. Observability Patterns All complex systems based on microservices have recommendations and patterns. This allows us to build a reliable system without reinventing the wheel. Observability systems also have some essential patterns. The following sections discuss five of the most important patterns. Log Aggregation Pattern In distributed systems, logging can be difficult. Each microservice can produce a lot of logs, and it can be a nightmare to find and analyze errors or other log messages of each microservice. Therefore, the log aggregation pattern helps us here. It contains the central log aggregation service as a central log storage. Also, the service provides options to label, index, categorize, search, and analyze all logs. There are a few examples of log aggregation platforms like Grafana Loki, Splunk, Fluentd, and the ELK stack. Figure 1: Log aggregation pattern Health Check Pattern Imagine you have multiple services or microservices, and you need to know their current state. Of course, you can go to the logging aggregation service and check logs. But services may not produce logs in a starting state. Also, it may be the case where logging in is unavailable when the services fail. In all these instances, you need to implement health check patterns. You just need to create a health (or ping) endpoint in your service and point your log aggregation system to check and collect the checks of each service. You can also set up notifications or alerting when the service is unavailable — it saves a lot of time recognizing what service failed to start or went down. Figure 2: Health check pattern Distributed Tracing Pattern Imagine this scenario: you have multiple components, modules, and libraries in one or several microservices. You need to check the whole history of component execution or send the request to one microservice, and you need to check the execution history from one service component list to another. To do this, you need to have some distributed system that will collect and analyze all tracing data. Some open-source services allow you to do so, such as Jaeger, OpenTelemetry, and OpenCensus. Check out the Istio documentation for an example that demonstrates distributed tracing in action. Figure 3: Distributed tracing pattern Application Metrics Pattern Having distributed logging and tracing is essential; however, without application metrics, your observability system will not be complete. You may need to collect infra- and application-level metrics, such as: CPU Memory Disc use Services requests/response time Latency Collecting these metrics will not only help you understand what infrastructure size you need but will also help you save money on cloud providers. It also helps you to quickly mitigate outages caused by a lack of CPU or memory resources. Below is an example of the service that has a proxy agent. The proxy agent aggregates and sends telemetry data to the observability platform. Figure 4: Application metrics pattern Service Mesh for Observability A service mesh not only provides a central management control plane for microservices architecture but also provides a single observability subsystem. Instead of installing a separate tool for gathering metrics, distributed traces, and logs, we can just use one. For example, Azure provides an integrated service mesh add-on that can be set up in a minute. There is also an option to use Istio service mesh, which contains all features required for a proper observability subsystem. Moreover, it can gather metrics, logs, and traces for the control plane. For example, when we set up Grafana, Loki, or other tools, we also need to enable observability for them, as they may also fail while working or during the deployment process; therefore, we need to troubleshoot. Figure 5: Service mesh as observability Observability Architecture for Microservices As an example of the observability architecture, I'm going to use a smart heating system. Smart heating is an essential part of each home (or even smart home) that allows owners to: Manually manage heating in the apartment with an application. Automatically adjust heating depending on time and the temperature outside and inside. In addition, the system can do the following actions to help the owner: Turn on/off the heating when people are about to arrive at the apartment. Notify, alert, or just ask if something requires human attention or if something is wrong. Figure 6: Microservices architecture with an observability subsystem In Figure 6, you can see an architecture that is based on the microservices pattern, since it serves best and represents all system components. It contains main and observability subsystems. Each microservice is based on Azure Functions and deployed to the Azure Kubernetes Cluster. We deploy functions to Kubernetes using the KEDA framework. KEDA is an open-source, Kubernetes-based event autoscaling that allows us to automatically deploy and scale our microservices functions. Also, KEDA provides the tools to wrap functions to the Docker containers. We can also deploy microservices functions directly without KEDA and Kubernetes if we don't have a massive load and don't need the scaling options. The architecture contains the following components that represent the main subsystem: Azure operating as a microservice Azure Service Bus (or Azure IoT Hub) as a central messaging bus that microservices use to communicate Azure API Apps providing an API for mobile/desktop applications The essential part here is an observability subsystem. A variety of components and tools represent it. I've described all components in Table 1 below: COMPONENTS OF THE OBSERVABILITY SYSTEM Tool Description Prometheus Prometheus is an open-source framework to collect and store logs and telemetry as time series data. Also, it provides alerting logic. Prometheus proxy or sidecar integrates with each microservice to collect all logs, telemetry, and tracing data. Grafana Loki Grafana Loki is an open-source distributed log aggregation service. It's based on a labeling algorithm. It's not indexing the logs; rather, it's assigning labels to each log domain, subsystem, or category. Jaeger Jaeger is an open-source framework for distributed tracing in microservices-based systems. It also provides search and data visualization options. Some of the high-level use cases of Jaeger include: Performance and latency optimization Distributed transaction monitoring Service dependency analysis Distributed context propagation Root cause analysis Grafana (Azure Managed Grafana) Grafana is also an open-source data visualization and analytics system. It allows the collection of traces, logs, and other telemetry data from different sources. We are using Grafana as a primary UI "control plane" to build and visualize data dashboards that will come from Prometheus, Loki, and Grafana Loki sources. Let's summarize our observability architecture. Logging is covered by Prometheus and Grafana Loki, and distributed tracing is covered by Jaeger. All these components report to Grafana, which provides UI data dashboards, analytics, and alerts. We can also use OpenTelemetry (OTel) framework. OTel is an open-source framework that was created, developed, and supported by the Cloud Native Computing Foundation (CNCF). The idea is to create a standardized vendor-free observability language specification, API, and tool. It is intended to collect, transform, and export telemetry data. Our architecture is based on the Azure cloud, and we can enable OpenTelemetry for our infrastructure and application components. Below you can see how our architecture can change with OpenTelemetry. Figure 7: Smart heating with an observability subsystem and OpenTelemetry It is also worth mentioning that we do not necessarily need to add OTel, as it may add additional complexity to the system. In the figure above, you can see that we need to forward all logs from Prometheus to OTel. Also, we can use Jaeger as a backend service for OTel. Grafana Loki and Grafana will get data from OTel. Conclusion In this article, we demystified observability and monitoring terms, and we walked through examples of microservices architecture with observability subsystems that can be used not only with Azure but also with other cloud providers. Also, we defined the main difference between monitoring and observability, and we walked through essential monitoring and observability patterns and toolsets. Developers and architects should understand that an observability/monitoring platform is a tooling or a technical solution that allows teams to actively debug their system. This is an article from DZone's 2022 Performance and Site Reliability Trend Report.For more: Read the Report

If we were that metaphorical fly on the wall, following an all too common Slack conversation between a software engineer and DevOps engineers, it might go something like this: Software Engineer: This is gonna take forever. “I need a new environment for my app.” Two hours later….. DevOps: Why do software engineers think I have telepathy!? “Okay, what instance types do you need?” An hour later (after consulting with the team…) Software Engineer: “I need a g3.8xlarge to test out our latest visualization feature.” Next day…. DevOps: “Cool, and what AZ do you need it in? Also, which security group should it be associated with?” Software Engineer: Don’t they know all this operational stuff, like automatically! “Any AZ in us-west-1, and it’s sg-3164z279.” 48 hours later after a few volleys back and forth over some additional parameters, permissions and other lovely details, DevOps gets the greenlight to spin up the environment, steps out to buy some licorice as a reward for all their suffering, and forgets to notify the software engineer that the request has been approved until 5 minutes before taking off for the day. Sound familiar? If so, there might be a highly unproductive cold war going on deep in the heart of your software engineering and DevOps departments. The Heart of the War What’s at the heart of this war? To understand that, let’s unpack two major issues that emerge from this not-so-smooth but all-too-familiar scenario. First, without a common language and clear communication channels, no two parties can work together even on simple tasks, let alone complex ones. Second, even with a common language, all the excess work, context switching, delays, and the inevitable friction, lead to cold-war-level frustration brewing within your organization. Adding to these issues are the blurred lines of responsibility that the DevOps model has created for both software engineering and DevOps (aka operations) teams. But the reality is that: Software engineers want to code, implement features and run them on infrastructure (so the customers can use them), without a lot of hassle and without getting bogged down in the operational details. DevOps want to focus on streamlining and keeping production stable, optimizing infrastructure, improving monitoring and general innovation, without getting sucked into the rabbit hole of end-user (e.g., software engineers’) service and access requests. When both sides spend multiple cycles on operational bottlenecks—in between throwing invisible daggers of hate at one another—the organization loses software development productivity as well as potential innovation from the DevOps side because nobody is getting what they want. This massive productivity loss can’t be measured by DORA metrics alone. It goes deep, right to the heart of your organization’s culture. But now, at last, there’s an end in sight to the decades-long cold war between software engineers and DevOps. Evolution of a DevOps Peace Prize Winner Nobody thinks the situation we’ve seen here—the radical disconnect between software engineers and DevOps—is okay. No business can work efficiently with this level of wasted time and effort. That’s why, a few years ago, insiders started proclaiming the dawn of self-service DevOps. When you think of self-service DevOps, it probably calls to mind little robots provisioning all the infrastructure your devs could possibly need. If only that were true. At the moment, self-service DevOps is still in its adolescent stage with cumbersome internal developer portals, service catalogs, workflow automation tools, and other shiny toys. But the space is rapidly maturing, thanks to an evolutionary process that’s already well underway. Yesterday: It Started With Chatbots Simulated conversation goes all the way back to the dawn of computing. Modern chatbots are definitely smarter but have still failed to live up to their initial promise, which was that chatbots would come to understand any request, easily integrate with DevOps tools, and automate workflows. In reality, chatbots are somewhat useful but face a number of fundamental problems. Essentially, they rely on simple, predetermined flows and rule-based, canned linear interactions. But we all know that in the real world, questions and requests can be varied and unique… essentially, not something a chatbot is equipped to handle. Plus, let’s face it—a chatbot that doesn’t do the job is worse than nothing at all: Software engineers try to guess the magic commands that will make the chatbot do their bidding, and if and when they fail (probably), they have to go running to the DevOps team anyway—except now it’s 24, 48, or 72 hours later, and they’re as frustrated as [fill in the blank]. To create an interface that will truly save time on the Engineering and Operations sides, you need more intelligence than a simple chatbot can provide. Today and Tomorrow: It Continues With Conversational AI Chatbots are limited because they don’t understand the languages that our (human) developers use. But what if you could use AI to power more sophisticated understanding? To achieve that level of understanding, you need two distinct strategies in place: Natural language processing (NLP) uses AI to systematically parse your words and, through extensive training as required by most AI solutions, tries to determine the meaning. Then, natural language understanding (NLU) goes one step further, learning to recognize variations in language that reflect the imprecise way that people communicate in the real world, including taking into consideration factors like sentiment, semantics, context, and intent. Essentially, NLP focuses on building algorithms to recognize and understand natural language, while NLU focuses on the meaning of a sentence. Putting these together, you finally arrive at true conversational AI. NLP and NLU are just one of a number of essential building blocks that go into conversational AI to provide genuine understanding and intelligence that will ultimately replace chatbots. Let’s look at some of those building blocks: Natural language processing engine (using NLP/NLU) to evaluate user input and understand what is being requested Integration with an identity provider (IDP) like Okta and other sources, such as a knowledge base or cloud providers, to keep tight control of permissions and security Iterative machine learning to identify new data sets and test user behavior predictions to drive continuous improvement of responses Dialog management system to retain context of the conversation and allow the conversational AI to respond accordingly Context management, or “keeping state,” to track exactly where the conversation left off and the last step that was reached Interface to interact with the user, usually via text or speech, ideally through their favorite workflow tool As an example of context management, in the fictionalized developer/DevOps conversation above, even after an interruption of 24 hours or longer, AI needs to remember what stage was reached so it can carry on once it receives authorization Furthermore, the last point—user interface—is critically important. To achieve true conversational intelligence that actually streamlines DevOps, you need an interface that meshes with the way your teams are already working. That way, you won’t add additional stress from context switching, which in itself is a big drain on productivity and focus. And the Peace Prize Winner Is… Your DevOps Virtual Assistant Remember the disconnect I mentioned earlier between Dev and DevOps? Well, a virtual assistant can bridge the gap, giving both sides exactly what they want—and need. Developers want to code, getting the infrastructure they need without a hassle. DevOps engineers want security and efficiency to avoid over-permissioning and excess cloud costs; they also don’t want to waste time on repetitive, tedious, tasks with lots of context switching. With a virtual assistant in place, here’s how the interaction might go: The software engineer uses their preferred work environment: Slack, Microsoft Teams, etc. They provide all the details in plain English. The virtual assistant then executes their request. For example: Provisions new cloud resources Triggers a complex workflow Provides some hard-to-find data Finally, the virtual assistant provides confirmation within the software engineer’s chosen work environment so they can get to work right away. With conversational AI, both sides stay focused and productive. Software engineers can focus on development, and DevOps won’t have to waste time on context switching or endless, repetitive requests. So they can all leave the building arm in arm at the end of the day, ready to work out those old resentments at the bowling alley. So let’s all hand conversational AI a Nobel Peace Prize; after decades of conflict, peace has broken out at last. And the big winner? Your organization, living in DevOps harmony happily ever after.

At this year’s Antwerp edition of the Devox Java conference, I attended some talks that managed to really inspire me, which is a rare gift. It made me think of my own 20+ year career as a developer and how I want/expect to spend the remainder. I like to share these thoughts here, giving credit to the excellent speakers where credit is due. Here are my three conclusions for the impatient: Been there, done that? I don’t buy it. You can never have been everywhere or done everything. The Java universe expands faster than anyone can keep up with, so don’t burn yourself out. The world needs excellent craftspeople more than it needs people to manage them. Don’t become a victim of the Peter Principle. If you aspire to a different job only for its status and are incompetent at it, you make more than one life miserable. Focus on Skills That Age Like Wine The biggest challenge is to stay relevant and sane in an ever-changing ecosystem. Choose your battles wisely, I say. Software products age like milk, not like wine or a Stradivarius violin. Language mastery gets rusty, however sharp your long-term memory. Your favorite JavaScript test framework has sprouted green hairs by the time you’re back from holidays. The half-life of technical relevancy is to two to five years (Learning through tinkering by Tom Cools). Is it any wonder some throw in the towel from pure frustration in trying to keep up? Apparently, the average developer is 28 and leaves the trade after 8 years, according to a statistic quoted by Tobias Modig (Get old, go slow, write code). Standard deviation is likely to be high, judging from the more senior attendees in the audience. So, take it with a pinch of salt. Yet whatever the exact numbers of this supposed exodus, burnout is a likely reason. For boredom, there is far less of an excuse. If you boast that the Java language has no more secrets, I dare you to take the latest OCP-17 exam unprepared. You will fail. It’s even worse when you try to take in the entire Java ecosystem. Have a look at the Developer Roadmap and try to stay optimistic. It’s obvious that you need a strategy of radical prioritization that respects work/life balance if you want to succeed at lifelong learning. I’m a fan of what I like to call just-in-time learning. I don’t go deep with a tool or technique until I can use it in a non-trivial project. And I don’t waste precious time mastering competing frameworks that are 90% complementary in features and purpose. Instead, I invest in the soft skills that do age like wine and are sorely underrated. Coding itself is only the tip of the creative iceberg. Coding is to development as inventing dialogue is to writing a play. The playwright needs to know the big picture and her target audience before she can let her fictional characters talk. Software is no different. There never existed any team where programmers were handed watertight specifications and turned it into flawless lines of code. It’s a Platonic ideal. Building software is a chaotic creative process where human language, not source code, turns out to be the biggest disruptor. It can take a decade on the job to get comfortable with that messy reality. Don’t Play the Status Game My second claim is that if tinkering with source code makes you happiest, you probably should not look for another job. Great employers value hard-earned technical experience from the trenches. They will not convince top developers to trade in a life of writing code for one sitting in meetings unless they really want it. Companies that have no enticing career path to offer their top technical talent other than management are gravely mistaken. They treat programming as an entry-level function that becomes unworthy of your experience after a while. They believe development is a young person’s game (like professional tennis), but they could not be more wrong. William Shakespeare wrote 37 plays over 25 years. Imagine he had been told after his first few successes: "Right, you’ve taken this as far as you can, time to move up the career ladder." There is no logical progression from coding to management any more than there is from playing the violin to conducting the orchestra. It’s a different profession, a different degree course, and even takes a different personality. Steve Jobs saw himself as the conductor of the Apple orchestra, an excellent analogy. If Apple were the New York Philharmonic, Jobs was like its legendary conductor Leonard Bernstein in temperament: a highly public, larger-than-life personality. Meanwhile, the musicians may not be recognized in the street, but each is at the top of their profession. Like Google, Apple, and Facebook, prestigious orchestras attract only top talent, and positions are highly sought after. But in the end, money and prestige don’t matter much if you like what you do and are respected for your skills. Excellent plumbers or plasterers delight their customers, coworkers, bosses, and not least themselves. So don’t conflate excellence with popularity or fame. Do you know Jony Ive? Probably not, as he cared more about his company’s brand than his own brand, but as chief designer his influence over the look and feel of Apple products was huge. If optimal visibility in the community is what you aim for, just realize that self-promoting your unique brand is a job in itself. With 10 million Java developers, that’s a lot of unique selling points to compete with. Tobias Modig was unhappy as a development manager. He had fallen victim to the Peter Principle and opted out. If your ambition drives you toward a career goal whose only appeal is its perceived status, it can happen to you too. You are promoted to your level of incompetence, constantly out of your depth, making more than one life miserable. This brings me to the last topic: you can’t excel at work that you hate, and you can’t enjoy work when you can’t do it well. Hitting the Sweet Spot of Your Career A satisfying career pays a decent wage and hits the ideal mix between what you like and what you’re good at. It’s an HR cliché, but no less true. Just realize that love doesn’t guarantee skill. I love playing the piano, but I’m still not great at it, even after many years. However, I see enough tangible progress to stay motivated. I couldn’t love something I’m terrible at. The road can be rough, but not getting anywhere is just an act of masochism. On the other hand, it’s hard to imagine ever being great at something you truly hated doing. If your heart is not in it, you stay mediocre at best, even when you’re capable of more, and that has probably happened to all of us. If you hate your job, for whatever reason (usually because of the people you work with), often the best and only fix is moving on, because you will stay miserable at anything you don’t like doing and therefore can’t do well. You will get no respect from your peers, and certainly not from your superiors for mere staying power. My conclusions are simple and modest. The happiest moment in any workweek is when I can sink my teeth into a juicy bit of coding or help a junior team member. Other than getting better at what I already do, I have no grander ambitions and if I can keep doing it for the next fifteen years for a paycheck, I’m happy and grateful. We are all a little vain and sensitive to the lure of status, but I will never switch jobs for the wrong reasons. Find out what you like and keep getting better at it. It’s more rewarding in the end.