How to do https request with bad certificate? [SOLVED]

Introduction

In the previous chapter, we introduced how to build an HTTPS server in Golang. In this post, we will answer the question “How to do an HTTPS request with a bad certificate in Golang?”. First of all, we need to remind what is the certificate in HTTPS protocol:

SSL certificates are what enable websites to move fromHTTPtoHTTPS, which is more secure. An SSL certificate is a data file hosted in a website’sorigin server. SSL certificates makeSSL/TLS encryptionpossible, and they contain the website’spublic keyand the website’s identity, along with related information. Devices attempting to communicate with the origin server will reference this file to obtain the public key and verify the server’s identity. The private key is kept secret and secure.

Some problems may cause the invalid certificate

• The site does not have an SSL certificate installed
• The SSL certificate has expired
• The SSL certificate is self-signed
• The SSL certificate is invalid (for example, the name of the domain does not match the name on the certificate).

Let’s see how to solve the problem of invalid SSL certificate.

Send HTTPS request using a bad certificate

Example 1: Request to a website with an expired SSL certificate

The example below will make some HTTPS requests to a website whose SSL certificate is expired:

package main

import (
    "log"
    "net/http"
)

func main() {
    // expired certificate
    _, err := http.Get("https://expired.badssl.com/")
    if err != nil {
        log.Fatal(err)
    }
}

Output:

2022/12/28 00:02:42 Get "https://expired.badssl.com/": x509: certificate has expired or is not yet valid: 
exit status 1

Example 2: Request to a website with a self-signed SSL certificate

The example below will make some HTTPS requests to a website whose self-signed SSL certificate:

package main

import (
    "log"
    "net/http"
)

func main() {
    // self-signed certificate
    _, err := http.Get("https://self-signed.badssl.com/")
    if err != nil {
        log.Fatal(err)
    }
}

Output:

2022/12/28 00:03:38 Get "https://self-signed.badssl.com/": x509: certificate signed by unknown authority
exit status 1

We saw that our attempt to make an HTTPS request using bad certificate using not working. Now let’s try to overcome this in next set of examples by turning of the security and making insecure connection.

Globally turn off the security checks for HTTPS Server

For all requests from the default client, you have the option to disable security checks by modifying the DefaultTransport. DefaultTransport is the default implementation of Transport and is used by DefaultClient. It establishes network connections as needed and caches them for reuse by subsequent calls.

defaultTransport := http.DefaultTransport.(*http.Transport)

// Create new Transport that ignores self-signed SSL
customTransport := &http.Transport{
  Proxy:                 defaultTransport.Proxy,
  DialContext:           defaultTransport.DialContext,
  MaxIdleConns:          defaultTransport.MaxIdleConns,
  IdleConnTimeout:       defaultTransport.IdleConnTimeout,
  ExpectContinueTimeout: defaultTransport.ExpectContinueTimeout,
  TLSHandshakeTimeout:   defaultTransport.TLSHandshakeTimeout,
  TLSClientConfig:       &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: customTransport}

We can set the TLSClientConfig with InsecureSkipVerify option to true to skip security checks as the code shown below:

package main

import (
    "crypto/tls"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "os"
)

func main() {

    http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
        // the self-signed certificate
    res, err := http.Get("https://self-signed.badssl.com/")
    if err != nil {
        log.Fatal(err)
    }

    resBody, err := ioutil.ReadAll(res.Body)
    if err != nil {
        fmt.Printf("client: could not read response body: %s\n", err)
        os.Exit(1)
    }
    fmt.Printf("client: response body: %s\n", resBody)
}

Output:

client: response body: <!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
  <link rel="apple-touch-icon" href="/icons/icon-red.png"/>
  <title>self-signed.badssl.com</title>
  <link rel="stylesheet" href="/style.css">
  <style>body { background: red; }</style>
</head>
<body>
<div id="content">
  <h1 style="font-size: 12vw;">
    self-signed.<br>badssl.com
  </h1>
</div>

</body>
</html>

Turn off security checks for client

With the same idea as the above example, in this example, we are going to disable the security check for a client:

package main

import (
    "crypto/tls"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "os"
)

func main() {
    tr := &http.Transport{
        TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
    }
    client := &http.Client{Transport: tr}
    // the expired certificate
    res, err := client.Get("https://expired.badssl.com/")
    if err != nil {
        log.Fatal(err)
    }

    resBody, err := ioutil.ReadAll(res.Body)
    if err != nil {
        fmt.Printf("client: could not read response body: %s\n", err)
        os.Exit(1)
    }
    fmt.Printf("client: response body: %s\n", resBody)
}

Output:

client: response body: <!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
  <link rel="apple-touch-icon" href="/icons/icon-red.png"/>
  <title>expired.badssl.com</title>
  <link rel="stylesheet" href="/style.css">
  <style>body { background: red; }</style>
</head>
<body>
<div id="content">
  <h1 style="font-size: 12vw;">
    expired.<br>badssl.com
  </h1>
</div>

</body>
</html>

Summary

I need to emphasize that: We should not disable security checks in a production environment***.*** To resolve the certificate problem, we should update the system certificate store on the client to include the missing intermediate certificate or check the SSL on the server.

References

https://www.cloudflare.com/learning/ssl/what-is-an-ssl-certificate/
https://stackoverflow.com/questions/12122159/how-to-do-a-https-request-with-bad-certificate