Python socket bind windows

Время на прочтение10 мин

Количество просмотров84K

Картинка с сайта: habr.com

К концу руководства вы освоите основные функции и методы модуля Python socket, научитесь применять пользовательский класс для отправки сообщений и данных между конечными точками и работать со всем этим в собственных клиент-серверных приложениях. Материалом делимся к старту курса по 

Fullstack-разработке на Python

.

Сеть при этом может быть логической, локальной сетью компьютера или физически подключённой к внешней сети, с собственными подключениями к другим сетям. Очевидный пример — интернет, к которому подключаются через провайдера.

Примеры протестированы на Python 3.10, но подойдёт и версия 3.6 или новее. Исходным код поможет использовать это руководство по максимуму.

Сети и сокеты — большие темы, по которым написаны томá литературы. Если они вам в новинку, переварить терминологию со всеми подробностями может быть трудно. Но с этим руководством всё получится!

История сокетов

История у сокетов давняя. Их применение началось с ARPANET в 1971 году и продолжилось в 1983-м, когда в операционной системе Berkeley Software Distribution (BSD) появился API под названием «сокеты Беркли».

В 1990-х годах вместе со Всемирной паутиной возникло и сетевое программирование. Преимущества сокетов и первых подключаемых сетей применялись не только в веб-серверах и браузерах — широко стали применяться клиент-серверные приложения разных типов и размеров.

Базовые протоколы API сокетов развивались многие годы, появились и новые, а низкоуровневый API остался прежним.

Самые распространённые сегодня приложения с сокетами — это клиент-серверные приложения, где одна сторона действует как сервер и ожидает подключения клиентов. Именно такое приложение вы напишете благодаря руководству. А конкретнее, сосредоточимся на API сокетов для интернет-сокетов. Иногда их называют сокетами Беркли, или сокетами BSD. Есть и сокеты домена Unix, которые используются для взаимодействия между процессами внутри только одного компьютера.

Обзор API сокетов

В модуле socket есть интерфейс к API сокетов Беркли.

Вот основные функции и методы этого API:

• .socket()
• .bind()
• .listen()
• .accept()
• .connect()
• .connect_ex()
• .send()
• .recv()
• .close()

В Python имеется удобный и последовательный API, напрямую сопоставленный с системными вызовами, то есть аналогами функций из списка выше на C. В следующем разделе вы узнаете, как эти функции используются вместе.

Кроме того, в стандартной библиотеке Python есть классы, которые упрощают применение этих функций. Хотя в этом руководстве socketserver не рассматривается, с этим фреймворком для сетевых серверов можно ознакомиться по ссылке.

Доступно много модулей, где реализованы интернет-протоколы уровня выше, например HTTP и SMTP. Обзор этих протоколов смотрите в разделе документации Python «Интернет-протоколы и их поддержка».

TCP-сокеты

С помощью socket.socket() вы создадите объект сокета с указанием типа сокета socket.SOCK_STREAM. При этом по умолчанию применяется протокол управления передачей (TCP). Возможно, это то, что вам нужно.

Но зачем вам TCP? Вот его особенности:

• TCP надёжен. Отброшенные в сети пакеты обнаруживаются и повторно передаются отправителем.
• Данные доставляются с сохранением порядка очерёдности. В приложении данные считываются в порядке их записи отправителем.

Для сравнения: сокеты, которые создаются через socket.SOCK_DGRAM протокола пользовательских датаграмм ненадёжны: данные могут считываться получателем с изменением порядка очерёдности записей отправителя. Почему это важно? Сети — это система негарантированной доставки. Нет гарантии, что данные дойдут до места назначения или что отправленные данные будут получены.

Сетевые устройства — маршрутизаторы и коммутаторы — также обладают конечной полосой пропускания и собственными, системными ограничениями. Как на клиентах и на серверах, у них есть процессоры, память, шины и интерфейсные буферы пакетов. С TCP при этом не нужно беспокоиться о потере пакетов, поступлении данных с изменением порядка очерёдности пакетов данных, а также о других подводных камнях.

Чтобы разобраться лучше, ознакомьтесь с последовательностью вызовов API сокетов и с потоком данных TCP.

Ниже слева сервер, а справа клиент:

Картинка с сайта: habr.com

Поток TCP-сокетов. В центре изображения показан обмен данными между клиентом и сервером с помощью вызовов .send() и .recv().

Внизу соответствующие сокеты закрываются на клиенте и на сервере. (источник изображения)

Начиная с верхнего левого угла, показаны серверные вызовы API на сервере, которые настраивают «прослушиваемый» сокет:

• socket()
• .bind()
• .listen()
• .accept()

Этим сокетом, как следует из его названия, прослушиваются подключения от клиентов. Чтобы принять или завершить такое подключение, на сервере вызывается .accept().

А чтобы установить подключение к серверу и инициировать трёхэтапное рукопожатие, на клиенте вызывается .connect(). Процесс рукопожатия важен, ведь он гарантирует доступность каждой стороны подключения в сети, то есть то, что клиент может связаться с сервером, и наоборот. Возможно, только один хост, клиент или сервер может связаться с другим.

Эхо-клиент и эхо-сервер

Теперь, когда вы узнали об API сокетов и взаимодействии клиента и сервера, вы готовы создать свои первые клиент и сервер. Начнём с примера, где полученное сервером сообщение просто возвращается клиенту, как эхо.

Эхо-сервер

Вот он:

# echo-server.py

import socket

HOST = "127.0.0.1"  # Standard loopback interface address (localhost)
PORT = 65432  # Port to listen on (non-privileged ports are > 1023)

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.bind((HOST, PORT))
    s.listen()
    conn, addr = s.accept()
    with conn:
        print(f"Connected by {addr}")
        while True:
            data = conn.recv(1024)
            if not data:
                break
            conn.sendall(data)

Не пытайтесь понять весь код сразу. В этих нескольких строках много чего происходит. И это только отправная точка, здесь можно увидеть базовый сервер в деле. Но что же происходит в вызове нашего API?

С помощью socket.socket() создаётся объект сокета, которым поддерживается тип контекстного менеджера, который используется в операторе with. Вызывать s.close() не нужно:

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    pass  # Use the socket object without calling s.close().

Передаваемые в socket() аргументы — это константы, используемые для указания семейства адресов и типа сокетов. AF_INET — это семейство интернет-адресов для IPv4. SOCK_STREAM — это тип сокета для TCP и протокол, который будет использоваться для передачи сообщений в сети.

Метод .bind() применяется для привязки сокета к конкретному сетевому интерфейсу и номеру порта:

# echo-server.py

# ...

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.bind((HOST, PORT))
    # ...

Передаваемые в .bind() значения зависят от семейства адресов сокета. В этом примере используется socket. AF_INET (IPv4). Поэтому принимается кортеж с двумя значениями: (host, port).

host может быть именем хоста, IP-адресом или пустой строкой. Если используется IP-адрес, то host должен быть строкой адреса формата IPv4. IP-адрес 127.0.0.1 — это стандартный IPv4-адрес для интерфейса «внутренней петли», когда к серверу подключаются только процессы в хосте. Если передавать пустую строку, подключения на сервере принимаются во всех доступных интерфейсах IPv4.

port — это номер TCP-порта для приёма подключений от клиентов. Это должно быть целое число от 1 до 65535(0 резервируется). В некоторых системах, если номер порта меньше 1024, могут потребоваться привилегии суперпользователя.

Относительно использования имён хостов с .bind() есть замечание:

«Если в хостовой части адреса сокета IPv4/v6 использовать имя хоста, программа может стать непредсказуемой: Python использует первый возвращаемый из разрешения DNS адрес. Адрес сокета будет разрешён в фактический адрес IPv4/v6 по-разному, в зависимости от результатов из DNS-разрешения и/или конфигурации хоста. Чтобы поведение было предсказыемым, в хостовой части используйте числовой адрес». Документация.

Подробнее об этом вы узнаете позже в разделе «Использование имён хостов». А пока достаточно понять, что при использовании имени хоста можно увидеть разные результаты в зависимости от того, чтó возвращается в процессе разрешения имён. Это может быть что угодно: при первом запуске приложения можно получить 10.1.2.3, а в следующий раз получится 192.168.0.1. Дальше может быть 172.16.7.8 и т. д.

В примере ниже подключения на сервере принимаются благодаря .listen(), а сам сервер становится «прослушиваемым» сокетом:

# echo-server.py

# ...

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.bind((HOST, PORT))
    s.listen()
    conn, addr = s.accept()
    # ...

У метода .listen() есть параметр backlog. Он указывает число непринятых подключений, которые система разрешит до отклонения новых подключений. С версии Python 3.5 он необязателен. Если его нет, выбирается значение backlog по умолчанию.

А если на сервере получается много одновременных запросов на подключение, значение backlog можно увеличить через установку максимальной длины очереди для отложенных подключений. Это предельное значение зависит от системы. Например, на Linux смотрите /proc/sys/net/core/somaxconn.

Методом .accept() выполнение блокируется, и ожидается входящее подключение. При подключении клиента возвращается новый объект сокета, который представляет собой подключение и кортеж с адресом клиента. В кортеже содержится (host, port) — для подключений IPv4 или (host, port, flowinfo, scopeid) — для IPv6. Подробнее о значениях кортежей рассказывается в справочном разделе «Семейства адресов сокетов».

Итак, теперь у вас есть новый объект сокета из .accept(). Это важно потому, что сокет будет использоваться для взаимодействия с клиентом. Он отличается от прослушиваемого, который применяется на сервере для приёма новых подключений:

# echo-server.py

# ...

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.bind((HOST, PORT))
    s.listen()
    conn, addr = s.accept()
    with conn:
        print(f"Connected by {addr}")
        while True:
            data = conn.recv(1024)
            if not data:
                break
            conn.sendall(data)

После того как в .accept() клиенту предоставляется объект сокета conn, для перебора блокирующих вызовов в conn.recv() используется бесконечный цикл while. Так любые отправляемые от клиента данные считываются и передаются обратно с помощью conn.sendall().

Если в conn.recv() возвращается пустой объект bytes и b'', значит, в клиенте подключение закрыто и цикл завершён. Чтобы автоматически закрыть сокет в конце блока, с conn применяется оператор with.

Эхо-клиент

Перейдём к клиенту:

# echo-client.py

import socket

HOST = "127.0.0.1"  # The server's hostname or IP address
PORT = 65432  # The port used by the server

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.connect((HOST, PORT))
    s.sendall(b"Hello, world")
    data = s.recv(1024)

print(f"Received {data!r}")

По сравнению с сервером клиент довольно прост. В нём создаётся объект сокета. Для подключения к серверу используется .connect(), и для отправки сообщения вызывается s.sendall(), s.recv() считывает ответ, а затем этот ответ выводится.

Запуск эхо-клиента и эхо-сервера

В этом разделе запускаем клиент и сервер, следим за их поведением и за происходящим.

> Если вам не удаётся запустить из командной строки примеры или собственный код, прочитайте How Do I Make My Own Command-Line Commands Using Python? или How to Run Your Python Scripts (англ.). Если у вас Windows, ознакомьтесь с Python Windows FAQ («Часто задаваемыми вопросами по Python для Windows»).

Откройте терминал или командную строку, перейдите в каталог со скриптами, убедитесь, что в переменной PATH у вас есть Python 3.6 или новее, а затем запустите сервер:

$ python echo-server.py

Терминал зависнет, потому что сервер заблокирован или находится в состоянии ожидания, в .accept():

# echo-server.py

# ...

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.bind((HOST, PORT))
    s.listen()
    conn, addr = s.accept()
    with conn:
        print(f"Connected by {addr}")
        while True:
            data = conn.recv(1024)
            if not data:
                break
            conn.sendall(data)

Ожидается подключение клиента. Затем откройте другое окно терминала или командную строку и запустите клиента:

$ python echo-client.py 
Received b'Hello, world'

В окне сервера вы должны заметить что-то такое:

$ python echo-server.py 
Connected by ('127.0.0.1', 64623)

Здесь на сервере выведен кортеж addr, возвращаемый из s.accept(). Это IP-адрес клиента и номер TCP-порта — 64623 (скорее всего, он будет другим, когда вы запустите сервер на своём компьютере).

Просмотр состояния сокета

Чтобы увидеть текущее состояние сокетов на хосте, используйте netstat. На macOS, Linux и Windows он доступен по умолчанию.

А ниже вывод netstat из macOS после запуска сервера:

$ netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp4       0      0  127.0.0.1.65432        *.*                    LISTEN

Обратите внимание: Local Address здесь 127.0.0.1.65432. Если бы в echo-server.py был HOST = "", а не HOST = "127.0.0.1", в netstat отображалось бы это:

$ netstat -an
Active Internet connections (including servers)
Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp4       0      0  *.65432                *.*                    LISTEN

Local Address здесь *.65432. Это означает, что для приёма входящих подключений будут задействованы все поддерживающие семейство адресов доступные интерфейсы хоста. В этом примере в вызове socket() используется socket. AF_INET (IPv4) — смотрите tcp4 в столбце Proto.

Здесь показывается только вывод эхо-сервера. Скорее всего, полный вывод будет гораздо больше, это зависит вашей системы. Стóит обратить внимание на столбцы Proto, Local Address и (state). В последнем примере netstat показывает, что на эхо-сервере используется TCP-сокет IPv4 (tcp4) в порте 65432 на всех интерфейсах (*.65432) и он находится в состоянии прослушивания (LISTEN).

Другой способ получить к нему доступ (и дополнительную полезную информацию) — использовать программу lsof, которая выводит список открытых файлов. На macOS она доступна по умолчанию, а на Linux её можно установить пакетным менеджером:

$ lsof -i -n
COMMAND     PID   USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
Python    67982 nathan    3u  IPv4 0xecf272      0t0  TCP *:65432 (LISTEN)

Если lsof используется с параметром -i, в её выводе предоставляется COMMAND, PID (идентификатор процесса) и USER (идентификатор пользователя) открытых интернет-сокетов. Выше показан процесс эхо-сервера.

netstat и lsof различаются в зависимости от ОС, у них много опций. Загляните в их man или документацию, на них определённо стóит потратить немного времени. На macOS и Linux используйте man netstat и man lsof. На Windows — netstat /? .

При попытке подключения к порту без прослушиваемого сокета есть типичная ошибка:

$ python echo-client.py 
Traceback (most recent call last):
  File "./echo-client.py", line 9, in <module>
    s.connect((HOST, PORT))
ConnectionRefusedError: [Errno 61] Connection refused

Здесь либо указан неверный номер порта, либо не запускается сервер. Или, может быть, на пути стоит брандмауэр, которым подключение блокируется (об этом легко забыть). Также может быть сообщение об ошибке Connection timed out («Превышено время ожидания подключения»). Чтобы клиент подключался к TCP-порту, добавьте соответствующее правило брандмауэра!

Поможем разобраться в программировании, чтобы вы прокачали карьеру или стали востребованным профессионалом в IT:

• Профессия Fullstack-разработчик на Python (15 месяцев)
• Профессия Data Scientist (24 месяца)

Чтобы увидеть все курсы, нажмите на баннер:

Картинка с сайта: habr.com

Сокеты. Создание сервера

Последнее обновление: 03.05.2023

Для создания сервера на языке Python, как и для клиента, также используется класс socket, однако общая работа будет несколько отличаться от работы с сокетом клиента.
Рассмотрим определение и работу с сокетом сервера поэтапно.

Привязка сервера

Сервер прослушивает входящие подключения, некоторым образом обрабатывает их и отправляет ответ. Чтобы сервер начал свою работу, вначале нам надо определить для него адрес, по которому
он будет прослушивать подключения. Для этого применяется метод bind()

socket.bind(address)

Данный метод принимает адрес, по которому будет запущен сервер. По умолчанию адрес представляет кортеж из двух элементов:

(host, port)

Первый элемент — хост в виде строки. Это может быть, например, IP-адрес в виде «127.0.0.1» или название локального хоста. Второй параметр — числовой номер порта. Порт представляет 2-х байтное значение от 0 до 65535.
Поскольку по одному и то же адресу (на одной и той же машине) может быть запущено несколько различных сетевых приложений, то порт позволяет разграничить эти приложения. Например:

import socket

server = socket.socket()            # создаем объект сокета сервера
hostname = socket.gethostname()     # получаем имя хоста локальной машины
port = 12345                        # устанавливаем порт сервера
server.bind((hostname, port))       # привязываем сокет сервера к хосту и порту

Здесь сервер будет запускаться на порту 12345. Следует учитывать, что не все порты могут быть свободны. Но, как правило, занятых портов не так много.

В качестве адреса используем имя текущего хоста. Для его получения применяется функция socket.gethostname() (обычно это имя текущего компьютера).

Прослушивание подключений

После привязки сервера его надо запустить на прослушивание подключений. Для этого применяется метод listen()

socket.listen([backlog])

Он принимает параметр backlog — максимальное количество входящих подключений в очереди, разрешенное для сокета. То есть, когда будут покдлючаться клиенты, они будут попадать в очередь и ждать,
пока сервер не обработает текущего клиента. Если в очереди уже есть указанное количество клиентов, ожидающих обработки сервером, то все новые клиенты отклоняются. Например:

import socket

server = socket.socket()            # создаем объект сокета сервера
hostname = socket.gethostname()     # получаем имя хоста локальной машины
port = 12345                        # устанавливаем порт сервера
server.bind((hostname, port))       # привязываем сокет сервера к хосту и порту
server.listen(5)    # начинаем прослушиваение входящих подключений

Получение и обработка клиента

Для получения входящих подключений применяется метод accept(). Этот метод возращает кортеж из двух элементов

(conn, address)

Первый элемент — conn представляет еще один объект socket, через который сервер взаимодействует с клиентом. Второй элемент — address — адрес
подключившегося клиента. Стоит отметить, что после завершения взаимодействия с клиентом сокет conn надо закрыть методом close()

Используя первый элемент кортежа — conn можно отправлять клиенту сообщения или наоборот получать данные. Для получения данных у сокета применяется метод socket.recv()

bytes = socket.recv(bufsize)

В качестве обязательного параметра он принимает максимальный размер буфера в байтах, которые могут быть получены за раз от другого сокета. Возвращаемое значение — набор байтов,
полученых от другого сокета.

Для отправки данных применяется метод socket.send(), который в качестве параметра получает набор отправляемых данных.

socket.send(bytes)

Теперь посмотрим все на примере. Пусть в файле server.py будет определен сервер со следующим кодом:

import socket

server = socket.socket()            # создаем объект сокета сервера
hostname = socket.gethostname()     # получаем имя хоста локальной машины
port = 12345                        # устанавливаем порт сервера
server.bind((hostname, port))       # привязываем сокет сервера к хосту и порту
server.listen(5)                    # начинаем прослушиваение входящих подключений

print("Server starts")

con, addr = server.accept()     # принимаем клиента
print("connection: ", con)
print("client address: ", addr)

message = "Hello Client!"       # сообщение для отправки клиенту
con.send(message.encode())      # отправляем сообщение клиенту
con.close()                     # закрываем подключение

print("Server ends")
server.close()

Здесь сервер принимает клиента, выводит информацию о его подключении и отправляет клиенту в ответ строку «Hello Client!».

А в файле client.py определим следующий код клиента:

import socket

client = socket.socket()            # создаем сокет клиента
hostname = socket.gethostname()     # получаем хост сервера
port = 12345                        # устанавливаем порт сервера
client.connect((hostname, port))    # подключаемся к серверу
data = client.recv(1024)            # получаем данные с сервера
print("Server sent: ", data.decode())    # выводим данные на консоль
client.close()                      # закрываем подключение

Поскольку у нас сервер и клиент будут запускать на одном и том же комптьютере, то для определения адреса сервера для подключения применяется функция socket.gethostname() и
порт 12345 — так же как и в коде сервера. После соединения с сервером получаем от него данные и выводим на консоль.

Сначала запустим код сервера. А затем запустим код клиента. В результате сервер примет подключение, выведет информацию о нем на консоль и отправит клиенту сообщение:

c:\python>python server.py
Server starts
connection:  <socket.socket fd=432, family=2, type=1, proto=0, laddr=('192.168.0.102', 12345), raddr=('192.168.0.102', 61824)>
client address:  ('192.168.0.102', 61824)
Server ends

В частности, здесь видим, что сервер и клиент запущены по адресу 192.168.0.102, причем клиент использует порт 61824.

А клиент получит от сервера сообщение и выведет его на консоль:

c:\python>python client.py
Server sent:  Hello Client!

Обработка множества клиентов

В данном случае сервер обслуживает одного клиента и прекращает работу. Если мы хотим, чтобы сервер обрабатывал множество клиентов, то мы можем использовать бесконечный цикл:

import socket
from datetime import datetime

server = socket.socket()            # создаем объект сокета сервера
hostname = socket.gethostname()     # получаем имя хоста локальной машины
port = 12345                        # устанавливаем порт сервера
server.bind((hostname, port))       # привязываем сокет сервера к хосту и порту
server.listen(5)                    # начинаем прослушиваение входящих подключений

print("Server running")
while True:
    con, addr = server.accept()     # принимаем клиента
    print("client address: ", addr)
    message = datetime.now().strftime("%H:%M:%S")  # отправляем текущее время
    con.send(message.encode())      # отправляем сообщение клиенту
    con.close()                     # закрываем подключение

В данном случае для примера с помощью встроенного модуля datetime и функции datetime.now().strftime()
получаем текущее время в виде строки, которая затем отправляется клиенту. В итоге при запросе клиент будет получать текущее время.

Двунаправленная связь

В примерах выше связь была однонаправленная — сервер отправлял данные, а клиент получал их. Рассмотрим простейшую двунаправленную связь, когда и клиент и сервер и отправляют, и получают данные.
Пусть сервер получает от клиента некоторую строку, инвертирует ее и отправляет обратно клиенту:

import socket

server = socket.socket()            # создаем объект сокета сервера
hostname = socket.gethostname()     # получаем имя хоста локальной машины
port = 12345                        # устанавливаем порт сервера
server.bind((hostname, port))       # привязываем сокет сервера к хосту и порту
server.listen(5)                    # начинаем прослушиваение входящих подключений

print("Server running")
while True:
    con, _ = server.accept()     # принимаем клиента
    data = con.recv(1024)           # получаем данные от клиента
    message = data.decode()         # преобразуем байты в строку
    print(f"Client sent: {message}")
    message = message[::-1]         # инвертируем строку
    con.send(message.encode())      # отправляем сообщение клиенту
    con.close()                     # закрываем подключение

А клиент пусть определяет код для ввода строки с консоли и ее отправки на сервер:

import socket

client = socket.socket()            # создаем сокет клиента
hostname = socket.gethostname()     # получаем хост локальной машины
port = 12345                        # устанавливаем порт сервера
client.connect((hostname, port))    # подключаемся к серверу
message = input("Input a text: ")   # вводим сообщение
client.send(message.encode())       # отправляем сообщение серверу
data = client.recv(1024)            # получаем данные с сервера
print("Server sent: ", data.decode()) 
client.close()                      # закрываем подключение

Результат работы. Клиент:

c:\python>python client.py
Input a text: hello
Server sent:  olleh

c:\python>

Сервер:

c:\python>python server.py
Server running
Client sent: hello

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Programming Sockets in Python

Socket programming is essential for network communication, enabling data exchange across different devices. In Python, sockets allow for inter-process communication (IPC) over networks. This tutorial provides a comprehensive guide on creating socket servers and clients, handling multiple connections, and managing errors in Python’s socket module.

By the end of this tutorial, you’ll understand that:

• A socket in Python is an endpoint for sending or receiving data across a network using the socket API.
• Socket programming in Python involves using sockets to establish communication between a server and clients over a network.
• A simple echo server in Python can be created using sockets to listen for client connections and echo back received messages.
• Handling multiple clients with Python sockets can be achieved using non-blocking sockets and the selectors module for concurrent connections.
• Connection errors in socket programs in Python can be managed by implementing error handling and using exceptions like OSError.

Along the way, you’ll learn about the main functions and methods in Python’s socket module that let you write your own client-server applications based on TCP sockets. You’ll learn how to reliably send messages and data between endpoints and handle multiple connections simultaneously.

Networking and sockets are large subjects. Literal volumes have been written about them. If you’re new to sockets or networking, it’s completely normal if you feel overwhelmed with all of the terms and pieces. To get the most out of this tutorial, it’s best to download the source code and have it on hand for reference while reading:

Take the Quiz: Test your knowledge with our interactive “Socket Programming in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

---

Картинка с сайта: realpython.com

Interactive Quiz

Socket Programming in Python

In this quiz, you’ll test your understanding of Python sockets. With this knowledge, you’ll be able to create your own client-server applications, handle multiple connections simultaneously, and send messages and data between endpoints.

TCP Sockets

You’re going to create a socket object using socket.socket(), specifying the socket type as socket.SOCK_STREAM. When you do that, the default protocol that’s used is the Transmission Control Protocol (TCP). This is a good default and probably what you want.

Why should you use TCP? The Transmission Control Protocol (TCP):

• Is reliable: Packets dropped in the network are detected and retransmitted by the sender.
• Has in-order data delivery: Data is read by your application in the order it was written by the sender.

In contrast, User Datagram Protocol (UDP) sockets created with socket.SOCK_DGRAM aren’t reliable, and data read by the receiver can be out-of-order from the sender’s writes.

Why is this important? Networks are a best-effort delivery system. There’s no guarantee that your data will reach its destination or that you’ll receive what’s been sent to you.

Network devices, such as routers and switches, have finite bandwidth available and come with their own inherent system limitations. They have CPUs, memory, buses, and interface packet buffers, just like your clients and servers. TCP relieves you from having to worry about packet loss, out-of-order data arrival, and other pitfalls that invariably happen when you’re communicating across a network.

To better understand this, check out the sequence of socket API calls and data flow for TCP:

Картинка с сайта: realpython.com

The left-hand column represents the server. On the right-hand side is the client.

Starting in the top left-hand column, note the API calls that the server makes to set up a “listening” socket:

• socket()
• .bind()
• .listen()
• .accept()

A listening socket does just what its name suggests. It listens for connections from clients. When a client connects, the server calls .accept() to accept, or complete, the connection.

The client calls .connect() to establish a connection to the server and initiate the three-way handshake. The handshake step is important because it ensures that each side of the connection is reachable in the network, in other words that the client can reach the server and vice-versa. It may be that only one host, client, or server can reach the other.

In the middle is the round-trip section, where data is exchanged between the client and server using calls to .send() and .recv().

At the bottom, the client and server close their respective sockets.

Communication Breakdown

Now you’ll take a closer look at how the client and server communicated with each other:

Картинка с сайта: realpython.com

When using the loopback interface (IPv4 address 127.0.0.1 or IPv6 address ::1), data never leaves the host or touches the external network. In the diagram above, the loopback interface is contained inside the host. This represents the internal nature of the loopback interface and shows that connections and data that transit it are local to the host.

Applications use the loopback interface to communicate with other processes running on the host and for security and isolation from the external network. Because it’s internal and accessible only from within the host, it’s not exposed.

You can see this in action if you have an application server that uses its own private database. If it’s not a database used by other servers, it’s probably configured to listen for connections on the loopback interface only. If this is the case, other hosts on the network can’t connect to it.

When you use an IP address other than 127.0.0.1 or ::1 in your applications, it’s probably bound to an Ethernet interface that’s connected to an external network. This is your gateway to other hosts outside of your “localhost” kingdom:

Картинка с сайта: realpython.com

Be careful out there. It’s a nasty, cruel world. Be sure to read the section Using Hostnames before venturing from the safe confines of “localhost.” There’s a security note that applies even if you’re not using hostnames but are using IP addresses only.

Application Client and Server

The multi-connection client and server example is definitely an improvement compared with where you started. However, now you can take one more step and address the shortcomings of the previous multiconn example in a final implementation: the application client and server.

You want a client and server that handle errors appropriately so that other connections aren’t affected. Obviously, your client or server shouldn’t come crashing down in a ball of fury if an exception isn’t caught. This is something you haven’t had to worry about until now, because the examples have intentionally left out error handling for brevity and clarity.

Now that you’re familiar with the basic API, non-blocking sockets, and .select(), you can add some error handling and address the elephant in the room, which the examples have kept hidden from you behind that large curtain over there. Remember that custom class that was mentioned way back in the introduction? That’s what you’re going to explore next.

First, you’ll address the errors:

All errors raise exceptions. The normal exceptions for invalid argument types and out-of-memory conditions can be raised; starting from Python 3.3, errors related to socket or address semantics raise OSError or one of its subclasses. (Source)

So, one thing you need to do is catch OSError. Another important consideration in relation to errors is timeouts. You’ll see them discussed in many places in the documentation. Timeouts happen and are a so-called normal error. Hosts and routers are rebooted, switch ports go bad, cables go bad, cables get unplugged, you name it. You should be prepared for these and other errors, handling them in your code.

What about the elephant in the room? As hinted by the socket type socket.SOCK_STREAM, when using TCP, you’re reading from a continuous stream of bytes. It’s like reading from a file on disk, but instead you’re reading bytes from the network. However, unlike reading a file, there’s no f.seek().

In other words, you can’t reposition the socket pointer, if there was one, and move around the data.

When bytes arrive at your socket, there are network buffers involved. Once you’ve read them, they need to be saved somewhere, or else you will have dropped them. Calling .recv() again reads the next stream of bytes available from the socket.

You’ll be reading from the socket in chunks. So, you need to call .recv() and save the data in a buffer until you’ve read enough bytes to have a complete message that makes sense to your application.

It’s up to you to define and keep track of where the message boundaries are. As far as the TCP socket is concerned, it’s just sending and receiving raw bytes to and from the network. It knows nothing about what those raw bytes mean.

This is why you need to define an application-layer protocol. What’s an application-layer protocol? Put simply, your application will send and receive messages. The format of these messages are your application’s protocol.

In other words, the length and format that you choose for these messages define the semantics and behavior of your application. This is directly related to what you learned in the previous paragraph regarding reading bytes from the socket. When you’re reading bytes with .recv(), you need to keep up with how many bytes were read, and figure out where the message boundaries are.

How can you do this? One way is to always send fixed-length messages. If they’re always the same size, then it’s easy. When you’ve read that number of bytes into a buffer, then you know you have one complete message.

However, using fixed-length messages is inefficient for small messages where you’d need to use padding to fill them out. Also, you’re still left with the problem of what to do about data that doesn’t fit into one message.

In this tutorial, you’ll learn a generic approach, one that’s used by many protocols, including HTTP. You’ll prefix messages with a header that includes the content length as well as any other fields you need. By doing this, you’ll only need to keep up with the header. Once you’ve read the header, you can process it to determine the length of the message’s content. With the content length, you can then read that number of bytes to consume it.

You’ll implement this by creating a custom class that can send and receive messages that contain text or binary data. You can improve and extend this class for your own applications. The most important thing is that you’ll be able to see an example of how this is done.

Before you get started, there’s something you need to know regarding sockets and bytes. As you learned earlier, when sending and receiving data via sockets, you’re sending and receiving raw bytes.

If you receive data and want to use it in a context where it’s interpreted as multiple bytes, for example a 4-byte integer, you’ll need to take into account that it could be in a format that’s not native to your machine’s CPU. The client or server on the other end could have a CPU that uses a different byte order than your own. If this is the case, then you’ll need to convert it to your host’s native byte order before using it.

This byte order is referred to as a CPU’s endianness. See Byte Endianness in the reference section for details. You’ll avoid this issue by taking advantage of Unicode for your message header and using the encoding UTF-8. Since UTF-8 uses an 8-bit encoding, there are no byte ordering issues.

You can find an explanation in Python’s Encodings and Unicode documentation. Note that this applies to the text header only. You’ll use an explicit type and encoding defined in the header for the content that’s being sent, the message payload. This will allow you to transfer any data that you’d like (text or binary), in any format.

You can easily determine the byte order of your machine by using sys.byteorder. For example, you could see something like this:

If you run this in a virtual machine that emulates a big-endian CPU (PowerPC), then something like this happens:

In this example application, your application-layer protocol defines the header as Unicode text with a UTF-8 encoding. For the actual content in the message, the message payload, you’ll still have to swap the byte order manually if needed.

This will depend on your application and whether or not it needs to process multi-byte binary data from a machine with a different endianness. You can help your client or server implement binary support by adding additional headers and using them to pass parameters, similar to HTTP.

Don’t worry if this doesn’t make sense yet. In the next section, you’ll see how all of this works and fits together.

Now you’ll fully define the protocol header. The protocol header is:

• Variable-length text
• Unicode with the encoding UTF-8
• A Python dictionary serialized using JSON

The required headers, or sub-headers, in the protocol header’s dictionary are as follows:

Name	Description
byteorder	The byte order of the machine (uses sys.byteorder). This may not be required for your application.
content-length	The length of the content in bytes.
content-type	The type of content in the payload, for example, text/json or binary/my-binary-type.
content-encoding	The encoding used by the content, for example, utf-8 for Unicode text or binary for binary data.

These headers inform the receiver about the content in the payload of the message. This allows you to send arbitrary data while providing enough information so that the content can be decoded and interpreted correctly by the receiver. Because the headers are in a dictionary, it’s easy to add additional headers by inserting key-value pairs as needed.

Sending an Application Message

There’s still a bit of a problem. You have a variable-length header, which is nice and flexible, but how do you know the length of the header when reading it with .recv()?

When you previously learned about using .recv() and message boundaries, you also learned that fixed-length headers can be inefficient. That’s true, but you’re going to use a small, 2-byte, fixed-length header to prefix the JSON header that contains its length.

You can think of this as a hybrid approach to sending messages. In effect, you’re bootstrapping the message receive process by sending the length of the header first. This makes it easy for your receiver to deconstruct the message.

To give you a better idea of the message format, check out a message in its entirety:

Картинка с сайта: realpython.com

A message starts with a fixed-length header of two bytes, which is an integer in network byte order. This is the length of the next header, the variable-length JSON header. Once you’ve read two bytes with .recv(), then you know you can process the two bytes as an integer and then read that number of bytes before decoding the UTF-8 JSON header.

The JSON header contains a dictionary of additional headers. One of those is content-length, which is the number of bytes of the message’s content (not including the JSON header). Once you’ve called .recv() and read content-length bytes, then you’ve reached a message boundary, meaning you’ve read an entire message.

Application Message Class

Finally, the payoff! In this section, you’ll study the Message class and see how it’s used with .select() when read and write events happen on the socket.

This example application reflects what types of messages a client and server could reasonably use. You’re far beyond toy echo clients and servers at this point!

To keep things simple and still demonstrate how things would work in a real application, this example uses an application protocol that implements a basic search feature. The client sends a search request and the server does a lookup for a match. If the request sent by the client isn’t recognized as a search, the server assumes it’s a binary request and returns a binary response.

After reading the following sections, running the examples, and experimenting with the code, you’ll see how things work. You can then use the Message class as a starting point and modify it for your own use.

The application is not that far off from the multiconn client and server example. The event loop code stays the same in app-client.py and app-server.py. What you’re going to do is move the message code into a class named Message and add methods to support reading, writing, and processing of the headers and content. This is a great example for using a class.

As you learned before and you’ll see below, working with sockets involves keeping state. By using a class, you keep all of the state, data, and code bundled together in an organized unit. An instance of the class is created for each socket in the client and server when a connection is started or accepted.

The class is mostly the same for both the client and the server for the wrapper and utility methods. They start with an underscore, like Message._json_encode(). These methods simplify working with the class. They help other methods by allowing them to stay shorter and support the DRY principle.

The server’s Message class works in essentially the same way as the client’s and vice-versa. The difference is that the client initiates the connection and sends a request message, followed by processing the server’s response message. Conversely, the server waits for a connection, processes the client’s request message, and then sends a response message.

It looks like this:

Step	Endpoint	Action / Message Content
1	Client	Sends a Message containing request content
2	Server	Receives and processes client request Message
3	Server	Sends a Message containing response content
4	Client	Receives and processes server response Message

Here’s the file and code layout:

Application	File	Code
Server	app-server.py	The server’s main script
Server	libserver.py	The server’s Message class
Client	app-client.py	The client’s main script
Client	libclient.py	The client’s Message class

With that, you should have a high-level overview of the individual components and their roles within the application.

Message Entry Point

Understanding how the Message class works can be a challenge because there’s an aspect of its design that might not be immediately obvious. Why? Managing state.

After a Message object is created, it’s associated with a socket that’s monitored for events using selector.register():

The key idea here is that each Message object is created when a new connection is accepted. It’s associated with a socket and registered with a selector to monitor for incoming events. This setup allows the server to handle multiple connections concurrently, ensuring that messages can be read as soon as they’re available.

When events are ready on the socket, they’re returned by selector.select(). You can then get a reference back to the message object using the data attribute on the key object and call a method in Message:

Looking at the event loop above, you’ll see that sel.select() is in the driver’s seat. It’s blocking, waiting at the top of the loop for events. It’s responsible for waking up when read and write events are ready to be processed on the socket. Which means, indirectly, it’s also responsible for calling the method .process_events(). That’s why .process_events() is the entry point.

Here’s what the .process_events() method does:

That’s good: .process_events() is simple. It can only do two things: call .read() and .write().

This is where managing state comes in. If another method depended on state variables having a certain value, then they would only be called from .read() and .write(). This keeps the logic as simple as possible as events come in on the socket for processing.

You might be tempted to use a mix of some methods that check the current state variables and, depending on their value, call other methods to process data outside .read() or .write(). In the end, this would likely prove too complex to manage and keep up with.

You should definitely modify the class to suit your own needs so that it works best for you. But, you’ll probably have the best results if you keep the state checks and the calls to methods that depend on that state to the .read() and .write() methods if possible.

Now look at .read(). This is the server’s version, but the client’s is the same. It just uses a different method name, .process_response() instead of .process_request():

The ._read() method is called first. It calls socket.recv() to read data from the socket and store it in a receive buffer.

Remember that when socket.recv() is called, all of the data that makes up a complete message may not have arrived yet. socket.recv() may need to be called again. This is why there are state checks for each part of the message before the appropriate method to process it is called.

Before a method processes its part of the message, it first checks to make sure enough bytes have been read into the receive buffer. If they have, it processes its respective bytes, removes them from the buffer and writes its output to a variable that’s used by the next processing stage. Because there are three components to a message, there are three state checks and process method calls:

Message Component	Method	Output
Fixed-length header	process_protoheader()	self._jsonheader_len
JSON header	process_jsonheader()	self.jsonheader
Content	process_request()	self.request

Next, check out .write(). This is the server’s version:

The .write() method checks first for a request. If one exists and a response hasn’t been created, .create_response() is called. The .create_response() method sets the state variable response_created and writes the response to the send buffer.

The ._write() method calls socket.send() if there’s data in the send buffer.

Remember that when socket.send() is called, all of the data in the send buffer may not have been queued for transmission. The network buffers for the socket may be full, and socket.send() may need to be called again. This is why there are state checks. The .create_response() method should only be called once, but it’s expected that ._write() will need to be called multiple times.

The client version of .write() is similar:

Because the client initiates a connection to the server and sends a request first, the state variable _request_queued is checked. If a request hasn’t been queued, it calls .queue_request(). The queue_request() method creates the request and writes it to the send buffer. It also sets the state variable _request_queued so that it’s only called once.

Just like for the server, ._write() calls socket.send() if there’s data in the send buffer. The notable difference in the client’s version of .write() is the last check to see if the request has been queued.

This will be explained more in the section Client Main Script, but the reason for this is to tell selector.select() to stop monitoring the socket for write events. If the request has been queued and the send buffer is empty, then you’re done writing and you’re only interested in read events. There’s no reason to be notified that the socket is writable.

To wrap up this section, consider this thought: the main purpose of this section was to explain that selector.select() is calling into the Message class via the method .process_events() and to describe how state is managed.

This is important because .process_events() will be called many times over the life of the connection. Therefore, make sure that any methods that should only be called once are either checking a state variable themselves, or the state variable set by the method is checked by the caller.

Server Main Script

In the server’s main script app-server.py, arguments are read from the command line that specify the interface and port to listen on:

For example, to listen on the loopback interface on port 65432, enter:

Use an empty string for <host> to listen on all interfaces.

After creating the socket, a call is made to socket.setsockopt() with the option socket.SO_REUSEADDR:

Setting this socket option avoids the error Address already in use. You’ll see this when starting the server on a port that has connections in the TIME_WAIT state.

For example, if the server actively closed a connection, it’ll remain in the TIME_WAIT state for two minutes or more, depending on the operating system. If you try to start the server again before the TIME_WAIT state expires, then you’ll get an OSError exception of Address already in use. This is a safeguard to make sure that any delayed packets in the network aren’t delivered to the wrong application.

The event loop catches any errors so that the server can stay up and continue to run:

When a client connection is accepted, a Message object is created:

The Message object is associated with the socket in the call to sel.register() and is initially set to be monitored for read events only. Once the request has been read, you’ll modify it to listen for write events only.

An advantage of taking this approach in the server is that in most cases, when a socket is healthy and there are no network issues, it’ll always be writable.

If you told sel.register() to also monitor EVENT_WRITE, then the event loop would immediately wake up and notify you that this is the case. However, at this point, there’s no reason to wake up and call .send() on the socket. There’s no response to send, because a request hasn’t been processed yet. This would consume and waste valuable CPU cycles.

Server Message Class

In the section Message Entry Point, you learned how the Message object was called into action when socket events were ready via .process_events(). Now you’ll learn what happens as data is read on the socket and a component, or piece, of the message is ready to be processed by the server.

The server’s message class is in libserver.py, which is part of the source code you downloaded earlier. You can also download the code by clicking the link below:

The methods appear in the class in the order in which processing takes place for a message.

When the server has read at least two bytes, the fixed-length header can be processed:

The fixed-length header is a 2-byte integer in network, or big-endian, byte order. It contains the length of the JSON header. You’ll use struct.unpack() to read the value, decode it, and store it in self._jsonheader_len. After processing the piece of the message it’s responsible for, .process_protoheader() removes it from the receive buffer.

Just like with the fixed-length header, when there’s enough data in the receive buffer to contain the JSON header, it can be processed as well:

The method self._json_decode() is called to decode and deserialize the JSON header into a dictionary. Because the JSON header is defined as Unicode with a UTF-8 encoding, utf-8 is hardcoded in the call. The result is saved to self.jsonheader. After processing the piece of the message that it’s responsible for, process_jsonheader() removes it from the receive buffer.

Next is the actual content, or payload, of the message. It’s described by the JSON header in self.jsonheader. When content-length bytes are available in the receive buffer, the request can be processed:

After saving the message content to the data variable, .process_request() removes it from the receive buffer. Then, if the content type is JSON, .process_request() decodes and deserializes it. If it’s not, this example application assumes that it’s a binary request and simply prints the content type.

The last thing .process_request() does is modify the selector to monitor write events only. In the server’s main script, app-server.py, the socket is initially set to monitor read events only. Now that the request has been fully processed, you’re no longer interested in reading.

A response can now be created and written to the socket. When the socket is writable, .create_response() is called from .write():

A response is created by calling other methods, depending on the content type. In this example application, a simple dictionary lookup is done for JSON requests when action == 'search'. For your own applications, you can define other methods that get called here.

After creating the response message, the state variable self.response_created is set so that .write() doesn’t call .create_response() again. Finally, the response is appended to the send buffer. This is seen by and sent via ._write().

One tricky bit to figure out is how to close the connection after the response is written. You can put the call to .close() in the method ._write():

Although it’s somewhat hidden, this is an acceptable trade-off given that the Message class only handles one message per connection. After the response is written, there’s nothing left for the server to do. It’s completed its work.

Client Main Script

In the client’s main script, app-client.py, arguments are read from the command line and used to create requests and start connections to the server:

Here’s an example:

After creating a dictionary representing the request from the command-line arguments, the host, port, and request dictionary are passed to .start_connection():

A socket is created for the server connection, as well as a Message object using the request dictionary.

Like for the server, the Message object is associated with the socket in the call to sel.register(). However, for the client, the socket is initially set to be monitored for both read and write events. Once the request has been written, you’ll modify it to listen for read events only.

This approach gives you the same advantage as the server: not wasting CPU cycles. After the request has been sent, you’re no longer interested in write events, so there’s no reason to wake up and process them.

Client Message Class

In the section Message Entry Point, you learned how the message object was called into action when socket events were ready via .process_events(). Now you’ll learn what happens after data is read and written on the socket and a message is ready to be processed by the client.

The client’s message class is in libclient.py, which is part of the source code you downloaded earlier. You can also download the code by clicking the link below:

The methods appear in the class in the order in which processing takes place for a message.

The first task for the client is to queue the request:

The dictionaries used to create the request, depending on what was passed on the command line, are in the client’s main script, app-client.py. The request dictionary is passed as an argument to the class when a Message object is created.

The request message is created and appended to the send buffer, which is then seen by and sent via ._write(). The state variable self._request_queued is set so that .queue_request() isn’t called again.

After the request has been sent, the client waits for a response from the server.

The methods for reading and processing a message in the client are the same as for the server. As response data is read from the socket, the process header methods are called: .process_protoheader() and .process_jsonheader().

The difference is in the naming of the final process methods and the fact that they’re processing a response, not creating one: .process_response(), ._process_response_json_content(), and ._process_response_binary_content().

Last, but certainly not least, is the final call for .process_response():

Okay. You can now wrap the message class up.

Message Class Wrapup

To conclude your learning about the Message class, it’s worth mentioning a couple of things that are important to notice with a few of the supporting methods.

Any exceptions raised by the class are caught by the main script in the except clause inside the event loop:

Note the line: message.close().

This is a really important line, for more than one reason! Not only does it make sure that the socket is closed, but message.close() also removes the socket from being monitored by .select(). This greatly simplifies the code in the class and reduces complexity. If there’s an exception or you explicitly raise one yourself, you know .close() will take care of the cleanup.

The methods Message._read() and Message._write() also contain something interesting:

Note the except BlockingIOError: line.

The ._write() method has one too. These lines are important because they catch a temporary error and skip over it using pass. The temporary error is when the socket would block, for example if it’s waiting on the network or the other end of the connection, also known as its peer.

By catching and skipping over the exception with pass, .select() will eventually trigger a new call, and you’ll get another chance to read or write the data.

Running the Application Client and Server

After all of this hard work, it’s time to have some fun and run some searches!

In these examples, you’ll run the server so that it listens on all interfaces by passing an empty string for the host argument. This will allow you to run the client and connect from a virtual machine that’s on another network. It emulates a big-endian PowerPC machine.

First, start the server:

Now run the client and enter a search. See if you can find him:

You might notice that the terminal is running a shell that’s using a text encoding of Unicode (UTF-8), so the output above prints nicely with emojis.

Now see if you can find the puppies:

Notice the byte string sent over the network for the request in the sending line. It’s easier to see if you look for the bytes printed in hex that represent the puppy emoji: \xf0\x9f\x90\xb6. If your terminal is using Unicode with the encoding UTF-8, you’ll be able to enter the emoji for the search.

This demonstrates that you’re sending raw bytes over the network and they need to be decoded by the receiver to be interpreted correctly. This is why you went to all of the trouble to create a header that contains the content type and encoding.

Here’s the server output from both client connections above:

Look at the sending line to see the bytes that were written to the client’s socket. This is the server’s response message.

You can also test sending binary requests to the server if the action argument is anything other than search:

Because the request’s content-type is not text/json, the server treats it as a custom binary type and doesn’t perform JSON decoding. It simply prints the content-type and returns the first ten bytes to the client:

If everything is working as expected, you’re all set! However, if you run into any issues along the way, don’t worry. Here’s some guidance to help you get back on track.

Troubleshooting

Inevitably, something won’t work, and you’ll be wondering what to do. Don’t worry, it happens to everyone. Hopefully, with the help of this tutorial, your debugger, and your favorite search engine, you’ll be able to get going again with the source code part.

If not, your first stop should be Python’s socket module documentation. Make sure you read all of the documentation for each function or method you’re calling. Also, read through the Reference section below for ideas. In particular, check the Errors section.

Sometimes, it’s not all about the source code. The source code might be correct, and it’s just the other host, the client, or server. Or it could be the network. Maybe a router, firewall, or some other networking device is playing man-in-the-middle.

For these types of issues, additional tools are essential. Below are a few tools and utilities that might help or at least provide some clues.

The ping Command

ping will check if a host is alive and connected to the network by sending an ICMP echo request. It communicates directly with the operating system’s TCP/IP protocol stack, so it works independently from any application running on the host.

Below is an example of running ping on macOS:

Note the statistics at the end of the output. This can be helpful when you’re trying to discover intermittent connectivity problems. For example, is there any packet loss? How much latency is there? You can check the round-trip times.

If there’s a firewall between you and the other host, a ping’s echo request may not be allowed. Some firewall administrators implement policies that enforce this. The idea is that they don’t want their hosts to be discoverable. If this is the case and you have firewall rules added to allow the hosts to communicate, then make sure that the rules also allow ICMP to pass between them.

ICMP is the protocol used by ping, but it’s also the protocol TCP and other lower-level protocols use to communicate error messages. If you’re experiencing strange behavior or slow connections, this could be the reason.

ICMP messages are identified by type and code. To give you an idea of the important information they carry, here are a few:

ICMP Type	ICMP Code	Description
8	0	Echo request
0	0	Echo reply
3	0	Destination network unreachable
3	1	Destination host unreachable
3	2	Destination protocol unreachable
3	3	Destination port unreachable
3	4	Fragmentation required, and DF flag set
11	0	TTL expired in transit

See the article Path MTU Discovery for information regarding fragmentation and ICMP messages. This is an example of something that can cause strange behavior.

The netstat Command

In the section Viewing Socket State, you learned how netstat can be used to display information about sockets and their current state. This utility is available on macOS, Linux, and Windows.

That section didn’t mention the columns Recv-Q and Send-Q in the example output. These columns will show you the number of bytes that are held in network buffers that are queued for transmission or receipt, but for some reason haven’t been read or written by the remote or local application.

In other words, the bytes are waiting in network buffers in the operating system’s queues. One reason could be that the application is CPU bound or is otherwise unable to call socket.recv() or socket.send() and process the bytes. Or there could be network issues affecting communications, like congestion or failing network hardware or cabling.

To demonstrate this and see how much data you can send before seeing an error, you can try out a test client that connects to a test server and repeatedly calls socket.send(). The test server never calls socket.recv(). It just accepts the connection. This causes the network buffers on the server to fill, which eventually raises an error on the client.

First, start the server:

Then run the client to see what the error is:

Here’s netstat output from while the client and server are still running, with the client printing out the error message above multiple times:

The first entry is the server (Local Address has port 65432):

Notice the Recv-Q: 408300.

The second entry is the client (Foreign Address has port 65432):

Notice the Send-Q: 269868.

The client sure was trying to write bytes, but the server wasn’t reading them. This caused the server’s network buffer queue to fill on the receive side and the client’s network buffer queue to fill on the send side.

Tools for Windows

If you work with Windows, there’s a suite of utilities that you should definitely check out if you haven’t already: Windows Sysinternals.

One of them is TCPView.exe. TCPView is a graphical netstat for Windows. In addition to addresses, port numbers, and socket state, it’ll show you running totals for the number of packets and bytes sent and received:

Картинка с сайта: realpython.com

Like with the Unix utility lsof, you also get the process name and ID. Check the menus for other display options.

Wireshark Network Analyzer

Sometimes you need to see what’s happening on the wire. Forget about what the application log says or what the value is that’s being returned from a library call. You want to see what’s actually being sent or received on the network. Just like with debuggers, when you need to see it, there’s no substitute.

Wireshark is a network protocol analyzer and traffic capture application that runs on macOS, Linux, and Windows, among others. There’s a GUI version named wireshark and also a terminal, text-based version named tshark.

Running a traffic capture is a great way to watch how an application behaves on the network and gather evidence about what it sends and receives, and how often and how much. You’ll also be able to see when a client or server closes or aborts a connection or stops responding. This information can be extremely helpful when you’re troubleshooting.

There are many good tutorials and other resources on the web that will walk you through the basics of using Wireshark and TShark.

Here’s an example of a traffic capture using Wireshark on the loopback interface:

Картинка с сайта: realpython.com

Here’s the same example shown above using tshark:

Next up, you’ll get more references to support your socket programming journey!

Frequently Asked Questions

Now that you have some experience with socket programming in Python, you can use the questions and answers below to check your understanding and recap what you’ve learned.

These FAQs are related to the most important concepts you’ve covered in this tutorial. Click the Show/Hide toggle beside each question to reveal the answer.

A socket in Python is an endpoint for sending or receiving data across a computer network. It allows for inter-process communication between applications on different machines or on the same machine. Python’s socket module provides a way to use the Berkeley sockets API to create and manage these connections.

Socket programming is a way to enable communication between two nodes on a network. In Python, it involves creating a socket object and using it to send and receive data. Python’s socket module includes methods to create client-server applications, handle connections, and manage data exchange using TCP or UDP protocols.

To write a simple echo server in Python, you need to create a socket, bind it to a specific host and port, and set it to listen for incoming connections. Once a connection is accepted, the server receives data from the client and sends it back, effectively echoing the data. The socket module provides functions and methods like socket(), .bind(), .listen(), .accept(), .recv(), and .sendall() to accomplish this.

To handle multiple clients with Python sockets, you can use the selectors module to manage multiple socket connections asynchronously. By registering sockets with a selector object, you can efficiently monitor them for read and write events, allowing you to handle each client connection in a non-blocking manner.

Connection errors in socket programs can be handled by catching exceptions such as OSError, ConnectionResetError, TimeoutError, and BlockingIOError. Implementing proper error handling ensures that your application can gracefully handle network issues, timeouts, and other connection-related problems without crashing.

Take the Quiz: Test your knowledge with our interactive “Socket Programming in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

---

Картинка с сайта: realpython.com

Interactive Quiz

Socket Programming in Python

In this quiz, you’ll test your understanding of Python sockets. With this knowledge, you’ll be able to create your own client-server applications, handle multiple connections simultaneously, and send messages and data between endpoints.

Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Programming Sockets in Python

Introduction

Socket programming is an essential part of network communication, and Python provides a robust library for it. Socket bind is one of the fundamental functions in socket programming, which is used to specify a port number for a socket. In this article, we will explore everything you need to know about socket bind in Python.

What is Socket Bind?

Socket bind is a function in socket programming that associates a socket with a specific IP address and a port number. In other words, it binds a socket to a specific address and port, so that other sockets can communicate with it. The socket bind function takes two arguments: the IP address and the port number.

Socket Bind in Python

Python provides a built-in module called ‘socket’ for socket programming. The ‘socket’ module contains various functions and classes for creating, manipulating, and communicating with sockets. The socket bind function in Python is implemented using the ‘bind’ method of the socket class.

Creating a Socket

Before we can use the socket bind function, we need to create a socket. In Python, we can create a socket using the ‘socket’ function of the ‘socket’ module. The ‘socket’ function takes two arguments: the address family and the socket type.

The address family specifies the type of addresses that the socket can communicate with. It can be either ‘AF_INET’ for IPv4 addresses or ‘AF_INET6’ for IPv6 addresses. The socket type specifies the type of communication that the socket will be used for. It can be either ‘SOCK_STREAM’ for TCP communication or ‘SOCK_DGRAM’ for UDP communication.

Here is an example of creating a socket in Python:

import socket
# Create a TCP/IP socketsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

In this example, we are creating a TCP/IP socket using the ‘socket’ function. We are specifying the address family as ‘AF_INET’ and the socket type as ‘SOCK_STREAM’.

Binding a Socket

Once we have created a socket, we can bind it to a specific IP address and port number using the socket bind function. In Python, we can use the ‘bind’ method of the socket class to bind a socket.

The ‘bind’ method takes one argument, which is a tuple containing the IP address and the port number. The IP address can be either a string representing the IP address or an empty string representing all available network interfaces. The port number can be any integer between 0 and 65535, but some ports are reserved for specific applications.

Here is an example of binding a socket in Python:

import socket
# Create a TCP/IP socketsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to a specific address and portserver_address = ('localhost', 8080)sock.bind(server_address)

In this example, we are binding a TCP/IP socket to the address ‘localhost’ and the port number ‘8080’. The address is specified as a tuple containing the IP address as a string and the port number as an integer.

Starting a Server

Once we have created and bound a socket, we can start a server that listens for incoming connections. In Python, we can use the ‘listen’ method of the socket class to start a server.

The ‘listen’ method takes one argument, which is the maximum number of queued connections. When a client tries to connect to the server, the server puts the client in a queue until it is ready to accept the connection.

Here is an example of starting a server in Python:

import socket
# Create a TCP/IP socketsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to a specific address and portserver_address = ('localhost', 8080)sock.bind(server_address)
# Start a server that listens for incoming connectionssock.listen(1)

In this example, we are starting a server that listens for incoming connections on the address ‘localhost’ and the port number ‘8080’. We are setting the maximum number of queued connections to 1.

Accepting a Connection

When a client tries to connect to the server, the server accepts the connection using the ‘accept’ method of the socket class. The ‘accept’ method returns a new socket object representing the connection and the address of the client.

Here is an example of accepting a connection in Python:

import socket
# Create a TCP/IP socketsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to a specific address and portserver_address = ('localhost', 8080)sock.bind(server_address)
# Start a server that listens for incoming connectionssock.listen(1)
# Accept a connectionconnection, client_address = sock.accept()

In this example, we are accepting a connection from a client. The ‘accept’ method returns a new socket object representing the connection and the address of the client. We are storing the socket object in the variable ‘connection’ and the client address in the variable ‘client_address’.

Closing a Socket

When we are done using a socket, we should close it using the ‘close’ method of the socket class. Closing a socket releases the resources used by the socket and prevents further communication with it.

Here is an example of closing a socket in Python:

import socket
# Create a TCP/IP socketsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to a specific address and portserver_address = ('localhost', 8080)sock.bind(server_address)
# Start a server that listens for incoming connectionssock.listen(1)
# Accept a connectionconnection, client_address = sock.accept()
# Close the socketconnection.close()sock.close()

In this example, we are closing the socket after accepting a connection from a client. We are using the ‘close’ method of the socket class to close the connection socket and the server socket.

Conclusion

Socket programming is an essential part of network communication, and Python provides a robust library for it. Socket bind is one of the fundamental functions in socket programming, which is used to specify a port number for a socket. In this article, we explored everything you need to know about socket bind in Python, including creating a socket, binding a socket, starting a server, accepting a connection, and closing a socket.

FAQ

1. What is a socket?

   A socket is an endpoint of a two-way communication link between two programs running on a network. It is identified by an IP address and a port number.

2. What is socket programming?

   Socket programming is a way of communicating between processes running on different machines over a network using sockets.

3. What is socket bind?

   Socket bind is a function in socket programming that associates a socket with a specific IP address and a port number.

4. What is the ‘socket’ module in Python?

   The ‘socket’ module is a built-in module in Python that provides various functions and classes for creating, manipulating, and communicating with sockets.

5. What is the difference between TCP and UDP?

   TCP is a connection-oriented protocol that guarantees the delivery of data in the correct order. UDP is a connectionless protocol that does not guarantee the delivery of data or the order of delivery.

6. What is the maximum value of a port number?

   The maximum value of a port number is 65535.

1. Introduction to Socket Programming

1.1. What is Socket Programming?

Socket programming is a method used to enable communication between two devices over a network. It can involve communication within the same device or between different devices across the world. Sockets provide the mechanism to send and receive data between devices using networking protocols like TCP and UDP.

1.2. Importance of Socket Programming in Networking

Sockets are the backbone of networking. They allow the development of robust client-server applications like web servers, chat applications, and file transfer protocols. Understanding how to use sockets in Python can help you build networked applications that are both efficient and secure.

1.3. Overview of Python’s socket Module

Python’s socket module provides a simple interface to create and manage network connections. It supports both connection-oriented (TCP) and connectionless (UDP) protocols. Here’s a simple example of how to create a socket:

import socket

# Creating a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

print("Socket created successfully") # Socket created successfully

2. Understanding the Basics of Sockets

2.1. What is a Socket?

A socket is an endpoint in a network communication channel. It consists of an IP address and a port number, which together uniquely identify a network service on a machine.

2.2. Types of Sockets

• TCP (Transmission Control Protocol): Reliable, connection-oriented communication.
• UDP (User Datagram Protocol): Faster, connectionless communication.

2.3. Socket Address Families

• IPv4 (AF_INET): Most commonly used IP address family.
• IPv6 (AF_INET6): Used for more modern IP address architecture.

2.4. Socket Types

• Stream (SOCK_STREAM): Provides a reliable, two-way, connection-based byte stream.
• Datagram (SOCK_DGRAM): Provides connectionless, unreliable messages (datagrams).

2.5. Socket Connection Modes

• Blocking Mode: The socket operations block the execution until the operation completes.
• Non-blocking Mode: The socket operations return immediately without waiting.

3. Creating a Simple TCP Server and Client

In this section, we will walk through the steps to create a simple TCP server and client using Python’s socket module. TCP (Transmission Control Protocol) is a connection-oriented protocol, meaning it requires a connection to be established between the server and client before data can be exchanged.

3.1. Steps to Create a TCP Server in Python

Creating a TCP server involves the following steps:

1. Create a socket: The server needs a socket to communicate with clients.
2. Bind the socket: Assign the socket to a specific IP address and port.
3. Listen for connections: The server waits for incoming connection requests.
4. Accept a connection: Once a connection is requested, the server accepts it.
5. Send and receive data: The server and client exchange data.
6. Close the connection: After communication is done, the connection is closed.

3.2. TCP Server Example

Let’s create a simple TCP server in Python:

import socket

# Step 1: Create a TCP/IP socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Step 2: Bind the socket to an address and port
server_socket.bind(('localhost', 65432))

# Step 3: Listen for incoming connections
server_socket.listen()

print("Server is listening on port 65432...")

# Step 4: Accept a connection
while True:
    client_socket, client_address = server_socket.accept()
    print(f"Connection from {client_address} has been established.")

    # Step 5: Send and receive data
    client_socket.sendall(b'Hello, client! Welcome to the server.')
    
    # Receive data from the client (optional, depending on use case)
    data = client_socket.recv(1024)
    print(f"Received {data.decode()} from {client_address}")

    # Step 6: Close the connection
    client_socket.close()

Output:

Server is listening on port 65432...

3.3. Steps to Create a TCP Client in Python

Creating a TCP client involves the following steps:

1. Create a socket: The client needs a socket to communicate with the server.
2. Connect to the server: The client connects to the server’s IP address and port.
3. Send and receive data: The client and server exchange data.
4. Close the connection: After communication is complete, the client closes the connection.

3.4. TCP Client Example

Let’s create a simple TCP client in Python:

import socket

# Step 1: Create a TCP/IP socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Step 2: Connect to the server
server_address = ('localhost', 65432)
client_socket.connect(server_address)

try:
    # Step 3: Send and receive data
    message = b'Hello, Server!'
    client_socket.sendall(message)

    # Wait for a response from the server
    data = client_socket.recv(1024)
    print(f"Received {data.decode()} from the server")

finally:
    # Step 4: Close the connection
    client_socket.close()

Output:

Received Hello, client! Welcome to the server. from the server

Meanwhile, the server will display:

Connection from ('127.0.0.1', <random port>) has been established.
Received Hello, Server! from ('127.0.0.1', <random port>)

3.5. Understanding the Server-Client Communication

• Server: The server waits for incoming client connections and establishes a connection when a client requests one. It can then exchange messages with the client over this connection. The server must run continuously to accept new connections.
• Client: The client initiates the connection by specifying the server’s address and port. Once connected, the client can send data to the server and receive responses. After the communication is complete, the client closes the connection.

4. Creating a Simple UDP Server and Client

UDP (User Datagram Protocol) is a connectionless protocol that allows the exchange of messages (datagrams) without establishing a connection. This makes UDP faster but less reliable than TCP, as it does not guarantee the delivery, order, or integrity of the messages.

In this section, we’ll walk through the process of creating a simple UDP server and client in Python. This example will demonstrate how to send and receive messages between the server and the client.

4.1. Steps to Create a UDP Server in Python

1. Create a socket: Use the socket.socket() function with the SOCK_DGRAM parameter to create a UDP socket.
2. Bind the socket: Bind the socket to an IP address and port number using the bind() method.
3. Receive data: Use the recvfrom() method to receive data from a client.
4. Send a response: Use the sendto() method to send a response back to the client.

4.2. UDP Server Example

Let’s start by creating a simple UDP server that listens for incoming messages and sends back a response.

import socket

def udp_server(host='127.0.0.1', port=12345):
    # Create a UDP socket
    server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

    # Bind the socket to the address
    server_socket.bind((host, port))

    print(f"UDP server is listening on {host}:{port}...")

    while True:
        # Receive message from client
        message, client_address = server_socket.recvfrom(1024)  # Buffer size is 1024 bytes
        print(f"Received message from {client_address}: {message.decode()}")

        # Optionally, send a response back to the client
        response = "Message received"
        server_socket.sendto(response.encode(), client_address)

if __name__ == "__main__":
    udp_server()

Output:

UDP server is listening on 127.0.0.1:12345...

4.3. Steps to Create a UDP Client in Python

1. Create a socket using socket.socket() with SOCK_DGRAM.
2. Send data using sendto().
3. Receive a response using recvfrom().

4.4. UDP Client Example:

import socket

def udp_client(server_host='127.0.0.1', server_port=12345, message="Hello, UDP Server!"):
    # Create a UDP socket
    client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

    # Send message to the server
    client_socket.sendto(message.encode(), (server_host, server_port))
    print(f"Message sent to {server_host}:{server_port}")

    # Optionally, receive a response from the server
    response, server_address = client_socket.recvfrom(1024)  # Buffer size is 1024 bytes
    print(f"Received response from server: {response.decode()}")

    # Close the socket
    client_socket.close()

if __name__ == "__main__":
    udp_client()

Output:

Message sent to 127.0.0.1:12345
Received response from server: Message received

5. Socket Options and Configuration

Socket options and configuration in Python are crucial for controlling the behavior of network connections. Python’s socket module provides a variety of methods to set and get socket options. Here’s a brief guide on how to work with socket options and configurations in Python:

5.1. Basic Socket Creation

Before configuring options, you need to create a socket:

import socket

# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Create a UDP socket
udp_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

5.2. Setting Socket Options

You can set socket options using the setsockopt() method. This method allows you to modify the behavior of sockets at various levels, typically the socket level (socket.SOL_SOCKET), but also at protocol levels.

5.2.1. SO_REUSEADDR

Allows a socket to bind to an address that is in a TIME_WAIT state.  

sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

5.2.2. SO_BROADCAST

Allows the transmission of broadcast messages on the socket.

udp_sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)

5.2.3. SO_KEEPALIVE

Enables keepalive packets on TCP connections.

sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)

5.2.4. TCP_NODELAY

Disables the Nagle algorithm, allowing small packets to be sent immediately.

sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

5.2.5. SO_RCVBUF

Sets the size of the receive buffer.

sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 4096)

5.2.6. SO_SNDBUF

Sets the size of the send buffer.

sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 4096)

5.3. Getting Socket Options

You can retrieve the current value of a socket option using the getsockopt() method.

# Get the current value of the SO_RCVBUF option
recv_buf_size = sock.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF)
print(f"Receive buffer size: {recv_buf_size}")

5.4. Socket Configuration

Here’s an example of configuring a socket with multiple options:

import socket

# Create a TCP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Set various socket options
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

# Bind the socket to an address and port
sock.bind(('localhost', 8080))

# Start listening for incoming connections
sock.listen(5)

print("Server is listening on port 8080...")

5.5 Timeouts

You can set a timeout for blocking socket operations using the settimeout() method:

# Set a timeout of 5 seconds
sock.settimeout(5.0)

try:
    conn, addr = sock.accept()
except socket.timeout:
    print("Connection attempt timed out.")

6. Error Handling in Socket Programming

Error handling in socket programming is crucial to ensure that your application can gracefully manage network issues, invalid operations, or unexpected errors. Below are some common techniques and best practices for handling errors in socket programming using Python:

6.1. Using try-except Blocks

• Always wrap socket operations within try-except blocks to catch and handle exceptions.
• Common exceptions include socket.error, socket.timeout, and others from the socket module.

import socket

try:
    # Creating a socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    # Connecting to a remote server
    sock.connect(('example.com', 80))
    # Sending data
    sock.sendall(b'GET / HTTP/1.1\r\nHost: example.com\r\n\r\n')
    # Receiving data
    response = sock.recv(4096)
    print(response)
except socket.error as e:
    print(f"Socket error: {e}")
except Exception as e:
    print(f"General error: {e}")
finally:
    sock.close()

6.2. Handling Specific Exceptions

• Python’s socket module defines several specific exceptions. You can handle them individually for more granular control:

try:
    sock.connect(('example.com', 80))
except socket.gaierror:
    print("Address-related error connecting to server.")
except socket.herror:
    print("Error with the host.")
except socket.timeout:
    print("Connection timed out.")
except socket.error as e:
    print(f"Other socket error: {e}")

6.3. Checking for Connection Errors

• Use sock.connect_ex() which returns an error code instead of raising an exception. This can be useful if you want to handle connection errors without exceptions.

error_code = sock.connect_ex(('example.com', 80))
if error_code != 0:
    print(f"Failed to connect: {error_code}")

6.4. Graceful Shutdown

• Ensure that your socket is closed properly even if an error occurs. This can be done using the finally block as shown earlier.

try:
    # Socket operations
except socket.error as e:
    print(f"Socket error: {e}")
finally:
    sock.close()  # Ensure socket is always closed

6.5. Using ‘with’ Statement for Automatic Resource Management

• Python 3.2+ supports the with statement for socket objects, ensuring the socket is closed automatically after the block is executed.

import socket

try:
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        sock.connect(('example.com', 80))
        sock.sendall(b'GET / HTTP/1.1\r\nHost: example.com\r\n\r\n')
        response = sock.recv(4096)
        print(response)
except socket.error as e:
    print(f"Socket error: {e}")

6.6. Logging Errors

• Consider logging errors for later diagnosis. Python’s logging module is useful for this.

import logging

logging.basicConfig(level=logging.ERROR)

try:
    sock.connect(('example.com', 80))
except socket.error as e:
    logging.error(f"Socket error: {e}")

6.7. Retry Logic

• Implement retry logic if a socket operation is prone to transient failures, such as network issues.

import time

max_retries = 5
for i in range(max_retries):
    try:
        sock.connect(('example.com', 80))
        break
    except socket.error as e:
        print(f"Retrying... {i+1}/{max_retries}")
        time.sleep(2)
else:
    print("Failed to connect after several retries.")

7. Advanced Socket Programming Techniques

Advanced socket programming in Python involves techniques that go beyond basic client-server communication. These techniques are essential for developing robust, high-performance, and scalable network applications. Here are some of the advanced socket programming techniques in Python:

7.1. Non-blocking Sockets and select()

• Non-blocking Mode: In non-blocking mode, socket operations (like connect, send, recv) return immediately, without waiting for the operation to complete. This is useful for applications where waiting is not an option.
• select() Function: The select() function allows you to monitor multiple sockets at once. You can check which sockets are ready for reading, writing, or have an exceptional condition pending, enabling efficient multiplexing.

Example:

import socket
import select

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setblocking(0)
s.bind(('localhost', 8080))
s.listen(5)

inputs = [s]
outputs = []
while inputs:
    readable, writable, exceptional = select.select(inputs, outputs, inputs)
    for sock in readable:
        if sock is s:
            connection, client_address = sock.accept()
            connection.setblocking(0)
            inputs.append(connection)
        else:
            data = sock.recv(1024)
            if data:
                outputs.append(sock)
            else:
                inputs.remove(sock)
                sock.close()

7.2. Multithreading and Multiprocessing

• Multithreading: This involves using Python’s threading module to handle multiple clients in separate threads. This is suitable when tasks are I/O-bound.
• Multiprocessing: For CPU-bound tasks, Python’s multiprocessing module is preferred. Each process can handle a socket, allowing full utilization of multi-core processors.

Example (Threading):

import socket
from threading import Thread

def handle_client(client_socket):
    request = client_socket.recv(1024)
    client_socket.send(b'ACK')
    client_socket.close()

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("0.0.0.0", 9999))
server.listen(5)

while True:
    client, addr = server.accept()
    client_handler = Thread(target=handle_client, args=(client,))
    client_handler.start()

7.3. SocketServer Framework

• SocketServer Module: Python’s SocketServer module provides a high-level server framework that simplifies the creation of network servers. It supports TCP, UDP, Unix streams, and datagram sockets, along with threading and forking versions.

import socketserver

class MyTCPHandler(socketserver.BaseRequestHandler):
    def handle(self):
        self.data = self.request.recv(1024).strip()
        self.request.sendall(self.data.upper())

if __name__ == "__main__":
    HOST, PORT = "localhost", 9999
    server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
    server.serve_forever()

7.4. TLS/SSL for Secure Sockets

• SSL/TLS: Python’s ssl module allows wrapping sockets for secure communication using the SSL/TLS protocol. This is critical for any application that transmits sensitive data over the network.

Example:

import socket
import ssl

context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
bindsocket = socket.socket()
bindsocket.bind(('127.0.0.1', 10023))
bindsocket.listen(5)

while True:
    newsocket, fromaddr = bindsocket.accept()
    connstream = context.wrap_socket(newsocket, server_side=True)
    try:
        data = connstream.recv(1024)
        connstream.sendall(data)
    finally:
        connstream.shutdown(socket.SHUT_RDWR)
        connstream.close()

7.5. Broadcast and Multicast

• Broadcast: Broadcasting allows sending a message to all devices on a network. This is useful in applications like network discovery services.
• Multicast: Multicasting is similar but sends data to a group of subscribed receivers, making it more efficient for certain use cases.

Example (Broadcast):

import socket

UDP_IP = "127.0.0.1" # Broadcast address, usually 255.255.255.255 or fixed LAN address
UDP_PORT = 5005 # broadcast port

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
sock.bind((UDP_IP, UDP_PORT))

MESSAGE = b"Hello, World!"
sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))

7.6. Load Balancing

• Load Balancing: Distributing the load among several servers can help scale applications. Python can implement basic load balancing using multiprocessing, or by integrating with load balancers like HAProxy.

Example (Round-robin Load Balancing):

import socket

servers = [('localhost', 8001), ('localhost', 8002)]
index = 0

while True:
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server = servers[index]
    index = (index + 1) % len(servers)
    sock.connect(server)
    sock.sendall(b'Hello, Server')
    response = sock.recv(1024)
    print('Received:', response)
    sock.close()

8. Practical Applications of Socket Programming

Socket programming in Python has several practical applications across various domains. Here are nine key examples:

1. Chat Applications: You can build real-time chat applications, allowing multiple users to communicate with each other over a network using TCP or UDP sockets.
2. Web Servers: Python can be used to create simple web servers that listen for HTTP requests on a socket, process them, and respond with HTML pages or JSON data.
3. File Transfer: Socket programming enables the transfer of files between a client and server, making it possible to create custom file-sharing applications.
4. Remote Command Execution: Sockets can be used to create a client-server architecture where commands can be sent remotely and executed on a server, useful for system administration tasks.
5. IoT Communication: For Internet of Things (IoT) devices, sockets facilitate communication between devices and central servers, enabling remote control and monitoring.
6. Multiplayer Games: Online multiplayer games often use sockets for real-time communication between players and game servers, ensuring synchronized gameplay.
7. Network Monitoring Tools: You can create tools that monitor network traffic by capturing and analyzing data packets transmitted over sockets.
8. Peer-to-Peer Networks: Sockets allow the creation of peer-to-peer (P2P) networks where nodes can communicate directly without relying on a centralized server, useful in file sharing or distributed computing.
9. Database Servers: Custom database servers can be implemented where client applications send queries over a socket connection, and the server processes these queries and returns results.

9. Common Pitfalls and How to Avoid Them

Socket programming in Python can be challenging due to several common pitfalls. Here’s a summary of these pitfalls and how to avoid them:

9.1. Blocking Calls

• Pitfall: Socket operations like recv() and accept() are blocking by default, which can cause the program to hang if there’s no data or connection.
• Avoidance: Use non-blocking sockets with setblocking(0) or leverage select or asyncio to handle multiple connections efficiently.

9.2. Improper Handling of Data Buffers

• Pitfall: Not handling partial data or assuming that all data will be received in one go can lead to incomplete message processing.
• Avoidance: Always loop and accumulate data until the complete message is received. Use fixed-size headers or delimiters to indicate the end of a message.

9.3. Socket Resource Leaks

• Pitfall: Forgetting to close sockets can lead to resource exhaustion and «too many open files» errors.
• Avoidance: Always close sockets in a finally block or use a with statement to ensure proper cleanup.

9.4. Port Exhaustion

• Pitfall: Rapidly creating and closing connections without proper timeouts can exhaust available ports.
• Avoidance: Implement proper connection handling, reuse sockets when possible, and use appropriate timeouts with socket.settimeout().

9.5. Hard-Coding IPs and Ports

• Pitfall: Hard-coding IP addresses and ports limit flexibility and can cause issues during deployment.
• Avoidance: Use configuration files or environment variables to set IP addresses and ports dynamically.

9.6. Error Handling

• Pitfall: Ignoring or improperly handling exceptions can lead to unhandled errors and crashes.
• Avoidance: Implement robust error handling with try-except blocks, logging errors, and taking corrective actions where possible.

9.7. Concurrency Issues

• Pitfall: Improper management of multiple threads or processes can lead to race conditions and deadlocks.
• Avoidance: Use thread-safe data structures, locks, or higher-level concurrency models like asyncio to manage concurrency safely.

10. Security Considerations in Socket Programming

When dealing with socket programming in Python, there are several key security considerations to keep in mind:

1. Data Encryption: Always encrypt data transmitted over sockets to protect it from eavesdropping. Use SSL/TLS (via ssl module) to secure communication channels.
2. Input Validation: Validate all incoming data to prevent attacks like buffer overflow, injection attacks, and other malicious inputs. This is crucial when handling data from untrusted sources.
3. Authentication: Implement proper authentication mechanisms to ensure that the connection is established only with trusted entities. This can be done using certificates or tokens.
4. Error Handling: Handle exceptions and errors properly to avoid leaking sensitive information about the server or the underlying system, which could be exploited by attackers.
5. Resource Management: Avoid resource exhaustion attacks by limiting the number of connections, setting timeouts, and closing unused sockets promptly. This prevents Denial-of-Service (DoS) attacks.
6. Firewall and Network Security: Ensure that your server is behind a firewall, and restrict socket access to trusted IP ranges to minimize exposure to potential attackers.
7. Use Non-blocking Sockets: Consider using non-blocking sockets to handle connections more efficiently and reduce the risk of being tied up by slow clients, which could lead to a form of DoS.
8. Avoid Hardcoding IPs and Ports: Dynamically assign IP addresses and ports, or retrieve them from secure configurations, rather than hardcoding them in your code. This prevents easy targeting by attackers.

11. Conclusion

Socket programming in Python allows you to create networked applications by enabling communication between devices over a network. Using Python’s socket module, you can establish connections, send and receive data, and handle various network protocols like TCP and UDP. It’s commonly used for building client-server applications, where the server listens for incoming connections and the client connects to the server to exchange data. This capability is essential for developing real-time applications like chat programs, web servers, and file transfer systems.  

Also Read:

Requests in Python

HTTP requests and responses in python with httpClient

urllib in Python