others-how to solve 'Connection reset by peer' or 'Connection closed by foreign host' when connecting to docker container ?

1. Purpose

In this post, I would demo how to solve the connection errors when connecting to docker container.

2. Environment

  • Docker Server Version: 20.10.2
  • Kernel Version: 4.15.0-128-generic Operating System: Ubuntu 18.04.5 LTS OSType: linux Architecture: x86_64

3. The code and the problem

3.1 The RESTful web service inside the container

I am developing a simple flask web service(RESTful) for test purpose, this little app just do the following two things:

  • Exporting a web service at localhost’s port 8080
  • When user access the url ‘http://localhost:8080’, it should return a hello world json string

The core code is as follows:

app1.py

from flask import Flask
from flask_restful import Resource, Api

app = Flask(__name__)
api = Api(app)

class HelloWorld(Resource):
    def get(self):
        return {'hello': 'world'}

api.add_resource(HelloWorld, '/')

if __name__ == '__main__':
    app.run(debug=True, port=8080)

If you don’t know flask-restful, here is the simple introduction:

Flask-RESTful is an extension for Flask that adds support for quickly building REST APIs. It is a lightweight abstraction that works with your existing ORM/libraries. Flask-RESTful encourages best practices with minimal setup. If you are familiar with Flask, Flask-RESTful should be easy to pick up.

Just install it as follows:

pip install flask-restful

More information about flask-restful can be found here.

When I run it as follows:

python app1.py

I got this result, it indicates that the app is running.

 * Serving Flask app "app1" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://127.0.0.1:8080/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 717-204-539

Then we test the service via curl command:

curl http://localhost:8080/

we got this:

{
    "hello": "world"
}

3.2 The Dockerfile

Now I want to package the app as a docker image, and then run it as a docker container, so I must write a dockerfile to construct the docker image.

First, we define a file named requirements.txt to define the dependency requirements for pip command , which would be used inside the container.

requirements.txt

Flask==1.1.1
flask-restful=0.3.8

Dockerfile

FROM python:3.6-buster
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY app1.py .
CMD ["python","app1.py"]

In the above Dockerfile, we have done these steps:

  • This docker image is based on python:3.6-buster, which is an offical python image
  • The ‘WORKDIR’ defines our working directory inside the docker container, docker would create a directory named /app inside the docker container, and all the files copied would be put to this directory
  • Then we copy the requirements.txt and do the ‘pip install -r …’ to install the python dependencies for the app, the ‘-r’ option of pip is explained as follows:
-r, --requirement <file>¶
Install from the given requirements file. This option can be used multiple times.
  • Then we copy the app1.py to our working directory
  • At last, we execute the python command to start our application

3.3 Build and run the docker image

Run the ‘docker build ‘ command in the project’s root directory:

docker build -t app1 -f ./Dockerfile .

The above command build a docker image named ‘app1’.

Then we run the docker image :

docker run --name app1 -p 8080:8080 --restart no --rm --detach app1

The above docker command runs the image ‘app1’, create a container named ‘app1’, exported the network port 8080 to host port 8080.

3.4 The connection problem

After running the docker container, we can verify it’s running:

[email protected]:~# docker ps
CONTAINER ID   IMAGE     COMMAND            CREATED        STATUS        PORTS                    NAMES
ad7b82319916   app1      "python app1.py"   18 hours ago   Up 18 hours   0.0.0.0:8080->8080/tcp   app1

Check the logs of the service inside the container:

[email protected]:~/app-flask-restful# docker logs -f ad
 * Serving Flask app "app1" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://127.0.0.1:8080/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 160-222-920

Now we use ‘curl’ to connect to the docker container:

curl http://localhost:8080/

We get this error or exception:

curl: (56) Recv failure: Connection reset by peer

Or if we try to telnet the service’s port, we get this :

[email protected]:~# curl http://127.0.0.1:8080/
curl: (56) Recv failure: Connection reset by peer
[email protected]:~# telnet 127.0.0.1 8080
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Connection closed by foreign host.
[email protected]:~#

Why did this happen? I think it should run, but it does not work!

4. Debug

I am suspecting that the docker network caused the problem, so I’d like to change the docker network to ‘host’ , by default, docker use the ‘bridge’ network :

  • bridge: The default network driver. If you don’t specify a driver, this is the type of network you are creating. Bridge networks are usually used when your applications run in standalone containers that need to communicate. See bridge networks.
  • host: For standalone containers, remove network isolation between the container and the Docker host, and use the host’s networking directly. See use the host network.

If we switch to ‘host’ network, the service is bound to the host’s network interface directly.

4.1 Stop the old container

We stop the old container as follows:

docker stop <container-id>

4.2 Start the docker container with network host

[email protected]:~# docker run --network host -d app1
11dacad2f2b9650151dcfbbee15884341f73fab988d043279c601fe672038650

4.3 Test the connection

[email protected]:~# curl http://127.0.0.1:8080
{
    "hello": "world"
}

It works, so the problem is identifed, the service is working on its own, but it can not be accessed from outside(docker network bridge), so ,we should change our app, it should bind to ‘0.0.0.0’ instead of ‘localhost’.

0.0.0.0 has a couple of different meanings, but in this context, when a server is told to listen on 0.0.0.0 that means “listen on every available network interface”. The loopback adapter with IP address 127.0.0.1 from the perspective of the server process looks just like any other network adapter on the machine, so a server told to listen on 0.0.0.0 will accept connections on that interface too.

5. The Solution

5.1 Change the network bound policy

Now we should bind to the network interface ‘0.0.0.0’.

if __name__ == '__main__':
    app.run(debug=True, port=8080, host="0.0.0.0")

5.2 Repackage the docker image

docker build -t app1 -f ./Dockerfile .

Sending build context to Docker daemon  6.656kB
Step 1/6 : FROM python:3.6-buster
 ---> d2c8d8ff1eb5
Step 2/6 : WORKDIR /app
 ---> Running in 74301bc0025e
Removing intermediate container 74301bc0025e
 ---> 4cf5b17bd895
Step 3/6 : COPY requirements.txt .
 ---> b63a56f6b8ca
Step 4/6 : RUN pip install -r requirements.txt
 ---> Running in ca94aa9c0537
Collecting Flask==1.1.1
  Downloading Flask-1.1.1-py2.py3-none-any.whl (94 kB)
Collecting flask-restful==0.3.8
  Downloading Flask_RESTful-0.3.8-py2.py3-none-any.whl (25 kB)
Collecting itsdangerous>=0.24
  Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Jinja2>=2.10.1
  Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
Collecting Werkzeug>=0.15
  Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting click>=5.1
  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting six>=1.3.0
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting pytz
  Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)
Collecting aniso8601>=0.82
  Downloading aniso8601-9.0.1-py2.py3-none-any.whl (52 kB)
Collecting MarkupSafe>=0.23
  Downloading MarkupSafe-1.1.1-cp36-cp36m-manylinux2010_x86_64.whl (32 kB)
Installing collected packages: MarkupSafe, Werkzeug, Jinja2, itsdangerous, click, six, pytz, Flask, aniso8601, flask-restful
Successfully installed Flask-1.1.1 Jinja2-2.11.3 MarkupSafe-1.1.1 Werkzeug-1.0.1 aniso8601-9.0.1 click-7.1.2 flask-restful-0.3.8 itsdangerous-1.1.0 pytz-2021.1 six-1.15.0
Removing intermediate container ca94aa9c0537
 ---> 520bec4911cc
Step 5/6 : COPY app1.py .
 ---> bba5c08e863d
Step 6/6 : CMD ["python", "app1.py"]
 ---> Running in 95b36bb37990
Removing intermediate container 95b36bb37990
 ---> 9e322e9ea3e7
Successfully built 9e322e9ea3e7
Successfully tagged app1:latest

5.3 Rerun the docker container

[email protected]:~/app-flask-restful# docker run --name app1 -p 8080:8080 --restart no --rm --detach app1
efd04c269d87d78ea59d047b90daeda6e2351668f2dcfbf417887e112b2f9399
[email protected]:~/app-flask-restful#
[email protected]:~/app-flask-restful# docker logs efd
 * Serving Flask app "app1" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 188-194-998
[email protected]:~/app-flask-restful#

5.4 Test the connection again

[email protected]:~/app-flask-restful# docker ps
CONTAINER ID   IMAGE     COMMAND            CREATED              STATUS              PORTS                    NAMES
efd04c269d87   app1      "python app1.py"   About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp   app1
[email protected]:~/app-flask-restful# curl http://127.0.0.1:8080

Now it works.

6. Summary

In this post, we demonstrated how to solve the connection problem when connecting to a docker container, the key point is that the container’s service is bound to localhost or 127.0.0.1, which can not be accessed from outside, you should change the code or configuration to bind to ip address ‘0.0.0.0’.

-->