{"id":1320,"date":"2023-02-02T13:43:11","date_gmt":"2023-02-02T13:43:11","guid":{"rendered":"https:\/\/infobip.com\/developers\/?p=1320"},"modified":"2023-09-11T14:31:42","modified_gmt":"2023-09-11T14:31:42","slug":"we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong","status":"publish","type":"post","link":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong","title":{"rendered":"Testing HAProxy configurations is hard, deploying untested changes is harder"},"content":{"rendered":"\n

Deploying an untested change to production will always cause some amount of stress.\u202f <\/p>\n\n\n\n

Stress level increases with the importance of a system you are changing. If there is any seasonality in your traffic patterns,\u202fstress additionally increases during a “high season<\/strong>“. That may be daily peaks, high traffic periods of the year, the week of your company going public, or any similar seasonality. Every past incident you had when you deployed some change also affects your stress level. The more recent they are, the more stress they add.\u202f\u202f <\/p>\n\n\n\n

In my subjective experience,\u202fdeploying a tested change reduces stress to almost zero<\/strong>. Even with testing in place, incidents can and will happen, but you have a way to catch them sooner and not repeat them.\u202f \u202f\u202f <\/p>\n\n\n\n

Reduced stress is reason enough to put effort into testing changes, even the ones that seem hard or even impossible to test before they reach production.\u202f\u202f <\/p>\n\n\n\n

HAProxy, the testing nemesis<\/strong> <\/h2>\n\n\n\n

In\u202fInfobip<\/a>, we deploy more than 1000 different changes to production daily. <\/strong>Most of those changes are code, which can be, and is, tested. But there are also changes in network configurations, virtualization, or storage layers, which are much harder to test.\u202f\u202f <\/p>\n\n\n\n

Until recently, one of the things we considered impossible to test was HAProxy configurations<\/strong>. But, as you will read further down the article, we just weren’t thinking enough about it.\u202f\u202f <\/p>\n\n\n\n

All our inbound HTTP traffic passes through an L7 balancing layer where we use HAProxy. Over the years, we have grown to 40+ data centers, many of which have specific configurations. Mostly due to special client requirements, various migrations, and different product stacks available in specific DCs. It’s impossible\u202fto have a staging environment addressing all the discrepancies<\/strong>.\u202f <\/p>\n\n\n\n

Where we fell short before<\/strong> <\/h2>\n\n\n\n

We tested a change\u202fon a single staging environment<\/strong>\u202fwe had, manipulating the configuration to a similar state to the one we wanted to change in the production. This process was\u202fentirely dependent on the engineers making the change<\/strong>.\u202f <\/p>\n\n\n\n

There was no hard procedure in place that required you to test the change before production. We considered it common sense and preached it, but when someone created a pull request for the change, we never checked if it was tested.\u202f <\/p>\n\n\n\n

\u202fThe only requirement for a change to get deployed to production was\u202fan approved pull request by one of your fellow engineer<\/strong>s. A few incidents later, we increased this to\u202ftwo engineers<\/strong>\u202ffor the most important systems. We considered this a temporary measure until we thought of something better. Of course, this was a pure “hopium” based strategy.\u202f\u202f <\/p>\n\n\n\n

Here’s an example of one HAProxy-related incident:\u202f <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

This was a change that resulted in serious degradation of our platform. After this was applied, some of the requests on the api.infobip.com<\/a>\u202fendpoint were routed to the wrong backend. As seen in the picture, it was a simple change.\u202f\u202f <\/p>\n\n\n\n

Two engineers approved it<\/strong>, both thinking that HAProxy behaves differently than it does. <\/p>\n\n\n\n

One of them was a new hire. Imagine the stress they experienced. And all the flashbacks that occurred when the next change was needed.\u202f <\/p>\n\n\n\n

We decided to improve this process significantly.\u202f\u202f <\/p>\n\n\n\n

A new approach: Scratch only where it itches<\/strong> <\/h2>\n\n\n\n

At first, we thought that testing HAProxy configurations before production was inherent to the type of configuration and the way we use it.\u202f\u202f <\/p>\n\n\n\n

“But every<\/em> HAProxy configuration is unique to aspecific data center!”<\/em> <\/p>\n\n\n\n

“We can’t fake<\/em> responses of<\/em> all backends!”<\/em> <\/p>\n\n\n\n

“There is a lot of a<\/em> client-specific context<\/em> in our HAProxy configurations, you must understand it to do<\/em> it right!”<\/em> <\/p>\n\n\n\n

A few costly incidents over the years made us think harder to find a solution. And it wasn’t that hard at all to find it. During the process, we learned that while these statements are still true, they don’t make the configuration untestable.\u202f <\/strong><\/p>\n\n\n\n

Our HAProxy configuration consists of many access lists routing traffic to specific backends depending on some regexes. When we took a long hard look at the incidents, we realized around 90% were due to a misconfigured regex or wrong order of access lists.\u202f <\/p>\n\n\n\n

That’s when we realized\u202fwe didn’t need to test every line of the HAProxy configuration<\/strong>\u202fand make the testing process more complicated than it should be. We should just test the things we are having problems with! Scratch only where it itches!\u202f\u202f <\/p>\n\n\n\n

And this itching part can be tested! If we make it so. So, we did.\u202f <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Every change we make in HAProxy configuration now passes a series of tests.\u202f\u202f <\/p>\n\n\n\n

Someone still must approve it, but every change must also pass a series of tests. On some HAProxies,\u202fwe now run more than 3000 tests on every change.<\/strong>\u202fPart of them was auto-generated, and part of them was written manually.\u202f <\/p>\n\n\n\n

Pipeline stages<\/strong>\u202f <\/h2>\n\n\n\n

Every push to the git repository triggers a Jenkins build, which creates an ad-hoc docker environment, prepares the HAProxy configuration for testing, and runs all the tests.\u202f\u202f <\/p>\n\n\n\n

HAProxy directives that are not subjected to tests are replaced with generic values.\u202f\u202f <\/p>\n\n\n\n

If any test fails, we consider the build failed, and the change cannot be merged to the production branch.\u202f <\/p>\n\n\n\n

Testing suite<\/strong>\u202f <\/h2>\n\n\n\n

Hurl<\/strong>\u202f <\/h3>\n\n\n\n

We have tried a couple of tools for evaluating responses to HTTP requests, and we chose\u202fhurl<\/a>\u202fdue to its good assertion engine, simple file format, and test execution speed.\u202f\u202f <\/p>\n\n\n\n

Hurl is a command <\/em>line tool that runs HTTP requests defined in a simple plain text format.\u202f <\/p>\n\n\n\n

It can perform <\/em>requests, capture values, and evaluate queries on headers and <\/em>body response. It is very versatile: it can be used for both fetching dana <\/em>and testing HTTP sessions.\u202f <\/p>\n\n\n\n

It can assert both JSON response and HTTP response headers. Additionally, you can even test whole HTML pages or even Bytes Content.\u202f <\/p>\n\n\n\n

HTTP-responder<\/strong>\u202f <\/h3>\n\n\n\n

A simple web server that just responds (thus this “brilliant” name) HTTP 200 OK, reflecting everything it received. Every HTTP request header that the HTTP-responder receives is returned in a JSON structure to the client.\u202f\u202f <\/p>\n\n\n\n

During configuration preparation, all backend servers in the HAProxy configuration are pointed to the\u202fHTTP-responder.\u202fAlso, the backend name is injected as a header to every request.\u202f\u202f <\/p>\n\n\n\n

This enables us to check if request is routed to the expected backend.\u202f <\/p>\n\n\n\n

Or if a reverse proxy injects some required header.\u202f\u202f <\/p>\n\n\n\n

Or if it blocks the request with malicious content in HTTP headers like we configured it.\u202f\u202f <\/p>\n\n\n\n

Tests<\/strong>\u202f <\/h1>\n\n\n\n

Things that we test are:\u202f <\/p>\n\n\n\n

    \n
  • access lists and request routing\u202f\u202f \u202f\u202f <\/li>\n<\/ul>\n\n\n\n
      \n
    • HTTP response status codes\u202f <\/li>\n\n\n\n
    • HTTP request header injections\u202f <\/li>\n\n\n\n
    • HTTP response header injections\u202f \u202f \u202f\u202f <\/li>\n\n\n\n
    • HTTP header or URL path rewrites\u202f <\/li>\n<\/ul>\n\n\n\n

      Writing the tests<\/strong>\u202f <\/h3>\n\n\n\n

      To test the behavior of a reverse proxy, you just need to describe the desired result of a specific request.\u202f\u202f <\/p>\n\n\n\n

      For example, we have the following directive in our configuration:\u202f <\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n

      To translate, all requests on\u202ftest.api.infobip.com\u202f<\/em>endpoint with a path beginning with\u202f\/mobile\/<\/em>\u202fshould be routed to the\u202fiam-mobile<\/em>\u202fbackend.\u202f\u202f <\/p>\n\n\n\n

      Now we just write this sentence as a test.\u202f\u202f <\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n

      This test is executed on every change. So, if we introduce some faulty regex that would make the traffic on \/mobile end up on another backend, this test will fail.\u202f\u202f <\/p>\n\n\n\n

      Zero incidents achievement: Unlocked<\/strong> <\/h2>\n\n\n\n

      Introducing a testing pipeline to our configuration change process has improved the quality of our platform,\u202fand the number of related incidents dropped to zero<\/strong>. My stress level is also quite close to that number now.\u202f\u202f <\/p>\n\n\n\n

      I’d like to encourage you to check your change deployment processes and\u202ftry to find a spot for automatic testing.\u202f\u202f<\/strong> <\/p>\n\n\n\n

      Remember: You don’t have to test everything.\u202f <\/p>\n\n\n\n

      Test what matters.\u202f <\/p>\n","protected":false},"excerpt":{"rendered":"

      We always thought haproxy configurations were impossible to test. We were wrong.<\/p>\n","protected":false},"author":21,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[28,249,254,252],"tags":[146,202,75],"coauthors":[165],"class_list":["post-1320","post","type-post","status-publish","format-standard","hentry","category-blog-post","category-devops-and-security","category-engineering-practices","category-tools","tag-development","tag-qa","tag-testing"],"acf":[],"yoast_head":"\nHAProxy configurations: How we learned to test them<\/title>\n<meta name=\"description\" content=\"We always thought HAProxy configurations were impossible to test. Turns out we just weren't thinking hard enough.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"HAProxy configurations: How we learned to test them\" \/>\n<meta property=\"og:description\" content=\"We always thought HAProxy configurations were impossible to test. Turns out we just weren't thinking hard enough.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong\" \/>\n<meta property=\"og:site_name\" content=\"Infobip Developers Hub\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/infobip\/\" \/>\n<meta property=\"article:published_time\" content=\"2023-02-02T13:43:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-11T14:31:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png\" \/>\n<meta name=\"author\" content=\"Ivan Sumak\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"Testing HAProxy configurations is hard. Deploying untested changes is harder.\" \/>\n<meta name=\"twitter:description\" content=\"We always thought HAProxy configurations were impossible to test. Turns out we just weren't thinking hard enough.\" \/>\n<meta name=\"twitter:creator\" content=\"@InfobipDev\" \/>\n<meta name=\"twitter:site\" content=\"@InfobipDev\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ivan Sumak\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong\"},\"author\":{\"name\":\"Ivan Sumak\",\"@id\":\"https:\/\/www.infobip.com\/developers\/#\/schema\/person\/d3c98eca3926c643f55fd54696cd7998\"},\"headline\":\"Testing HAProxy configurations is hard, deploying untested changes is harder\",\"datePublished\":\"2023-02-02T13:43:11+00:00\",\"dateModified\":\"2023-09-11T14:31:42+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong\"},\"wordCount\":1310,\"publisher\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#primaryimage\"},\"thumbnailUrl\":\"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png\",\"keywords\":[\"development\",\"QA\",\"testing\"],\"articleSection\":[\"Blog Post\",\"DevOps and Security\",\"Engineering Practices\",\"Tools\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong\",\"url\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong\",\"name\":\"HAProxy configurations: How we learned to test them\",\"isPartOf\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#primaryimage\"},\"thumbnailUrl\":\"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png\",\"datePublished\":\"2023-02-02T13:43:11+00:00\",\"dateModified\":\"2023-09-11T14:31:42+00:00\",\"description\":\"We always thought HAProxy configurations were impossible to test. Turns out we just weren't thinking hard enough.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#primaryimage\",\"url\":\"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png\",\"contentUrl\":\"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.infobip.com\/developers\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Testing HAProxy configurations is hard, deploying untested changes is harder\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.infobip.com\/developers\/#website\",\"url\":\"https:\/\/www.infobip.com\/developers\/\",\"name\":\"Infobip Developers Hub\",\"description\":\"Build meaningful customer relationships across any channel\",\"publisher\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.infobip.com\/developers\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.infobip.com\/developers\/#organization\",\"name\":\"Infobip Developers Hub\",\"url\":\"https:\/\/www.infobip.com\/developers\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.infobip.com\/developers\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.infobip.com\/developers\/wp-content\/uploads\/2023\/03\/Infobip_logo_favicon.png\",\"contentUrl\":\"https:\/\/www.infobip.com\/developers\/wp-content\/uploads\/2023\/03\/Infobip_logo_favicon.png\",\"width\":696,\"height\":696,\"caption\":\"Infobip Developers Hub\"},\"image\":{\"@id\":\"https:\/\/www.infobip.com\/developers\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/infobip\/\",\"https:\/\/x.com\/InfobipDev\",\"https:\/\/www.youtube.com\/channel\/UCUPSTy53VecI5GIir3J3ZbQ\",\"https:\/\/github.com\/infobip-community\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.infobip.com\/developers\/#\/schema\/person\/d3c98eca3926c643f55fd54696cd7998\",\"name\":\"Ivan Sumak\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.infobip.com\/developers\/#\/schema\/person\/image\/9f184bb10b2725482ee819ead4f56860\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b4390fa2e8da460cc44d8c20bf69bf6ba65f6cefc7ab74c57358eb0e047b2bde?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b4390fa2e8da460cc44d8c20bf69bf6ba65f6cefc7ab74c57358eb0e047b2bde?s=96&d=mm&r=g\",\"caption\":\"Ivan Sumak\"},\"description\":\"Ivan is a software engineer with 15 years of experience in different fields of the IT ecosystem \u2014 Linux, DevOps practices, Kubernetes, cloud native. Any kind of automation of cumbersome processes makes him happy, almost as much as weekends in the mountains.\",\"sameAs\":[\"https:\/\/hr.linkedin.com\/in\/ivan-umak-8915491a\"],\"url\":\"https:\/\/www.infobip.com\/developers\/blog\/author\/ivan-sumak\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"HAProxy configurations: How we learned to test them","description":"We always thought HAProxy configurations were impossible to test. Turns out we just weren't thinking hard enough.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong","og_locale":"en_US","og_type":"article","og_title":"HAProxy configurations: How we learned to test them","og_description":"We always thought HAProxy configurations were impossible to test. Turns out we just weren't thinking hard enough.","og_url":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong","og_site_name":"Infobip Developers Hub","article_publisher":"https:\/\/www.facebook.com\/infobip\/","article_published_time":"2023-02-02T13:43:11+00:00","article_modified_time":"2023-09-11T14:31:42+00:00","og_image":[{"url":"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png","type":"","width":"","height":""}],"author":"Ivan Sumak","twitter_card":"summary_large_image","twitter_title":"Testing HAProxy configurations is hard. Deploying untested changes is harder.","twitter_description":"We always thought HAProxy configurations were impossible to test. Turns out we just weren't thinking hard enough.","twitter_creator":"@InfobipDev","twitter_site":"@InfobipDev","twitter_misc":{"Written by":"Ivan Sumak","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#article","isPartOf":{"@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong"},"author":{"name":"Ivan Sumak","@id":"https:\/\/www.infobip.com\/developers\/#\/schema\/person\/d3c98eca3926c643f55fd54696cd7998"},"headline":"Testing HAProxy configurations is hard, deploying untested changes is harder","datePublished":"2023-02-02T13:43:11+00:00","dateModified":"2023-09-11T14:31:42+00:00","mainEntityOfPage":{"@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong"},"wordCount":1310,"publisher":{"@id":"https:\/\/www.infobip.com\/developers\/#organization"},"image":{"@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#primaryimage"},"thumbnailUrl":"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png","keywords":["development","QA","testing"],"articleSection":["Blog Post","DevOps and Security","Engineering Practices","Tools"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong","url":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong","name":"HAProxy configurations: How we learned to test them","isPartOf":{"@id":"https:\/\/www.infobip.com\/developers\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#primaryimage"},"image":{"@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#primaryimage"},"thumbnailUrl":"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png","datePublished":"2023-02-02T13:43:11+00:00","dateModified":"2023-09-11T14:31:42+00:00","description":"We always thought HAProxy configurations were impossible to test. Turns out we just weren't thinking hard enough.","breadcrumb":{"@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#primaryimage","url":"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png","contentUrl":"https:\/\/infobip.com\/developers\/wp-content\/uploads\/2023\/02\/image.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.infobip.com\/developers\/blog\/we-always-thought-haproxy-configurations-were-impossible-to-test-we-were-wrong#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.infobip.com\/developers\/"},{"@type":"ListItem","position":2,"name":"Testing HAProxy configurations is hard, deploying untested changes is harder"}]},{"@type":"WebSite","@id":"https:\/\/www.infobip.com\/developers\/#website","url":"https:\/\/www.infobip.com\/developers\/","name":"Infobip Developers Hub","description":"Build meaningful customer relationships across any channel","publisher":{"@id":"https:\/\/www.infobip.com\/developers\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.infobip.com\/developers\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.infobip.com\/developers\/#organization","name":"Infobip Developers Hub","url":"https:\/\/www.infobip.com\/developers\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.infobip.com\/developers\/#\/schema\/logo\/image\/","url":"https:\/\/www.infobip.com\/developers\/wp-content\/uploads\/2023\/03\/Infobip_logo_favicon.png","contentUrl":"https:\/\/www.infobip.com\/developers\/wp-content\/uploads\/2023\/03\/Infobip_logo_favicon.png","width":696,"height":696,"caption":"Infobip Developers Hub"},"image":{"@id":"https:\/\/www.infobip.com\/developers\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/infobip\/","https:\/\/x.com\/InfobipDev","https:\/\/www.youtube.com\/channel\/UCUPSTy53VecI5GIir3J3ZbQ","https:\/\/github.com\/infobip-community"]},{"@type":"Person","@id":"https:\/\/www.infobip.com\/developers\/#\/schema\/person\/d3c98eca3926c643f55fd54696cd7998","name":"Ivan Sumak","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.infobip.com\/developers\/#\/schema\/person\/image\/9f184bb10b2725482ee819ead4f56860","url":"https:\/\/secure.gravatar.com\/avatar\/b4390fa2e8da460cc44d8c20bf69bf6ba65f6cefc7ab74c57358eb0e047b2bde?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b4390fa2e8da460cc44d8c20bf69bf6ba65f6cefc7ab74c57358eb0e047b2bde?s=96&d=mm&r=g","caption":"Ivan Sumak"},"description":"Ivan is a software engineer with 15 years of experience in different fields of the IT ecosystem \u2014 Linux, DevOps practices, Kubernetes, cloud native. Any kind of automation of cumbersome processes makes him happy, almost as much as weekends in the mountains.","sameAs":["https:\/\/hr.linkedin.com\/in\/ivan-umak-8915491a"],"url":"https:\/\/www.infobip.com\/developers\/blog\/author\/ivan-sumak"}]}},"_links":{"self":[{"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/posts\/1320","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/comments?post=1320"}],"version-history":[{"count":5,"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/posts\/1320\/revisions"}],"predecessor-version":[{"id":1939,"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/posts\/1320\/revisions\/1939"}],"wp:attachment":[{"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/media?parent=1320"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/categories?post=1320"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/tags?post=1320"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.infobip.com\/developers\/wp-json\/wp\/v2\/coauthors?post=1320"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}