Мой путь от python к go

Need Help?

Need help with any of this? Read BeginnersGuide/Help for mailing lists and newsgroups.


Most Python books will include an introduction to the language; see IntroductoryBooks for suggested titles.

Consult BeginnersGuide/Examples for small programs and little snippets of code that can help you learn.

Or, if you prefer to learn Python through listening to a lecture, you can attend a training course or even hire a trainer to come to your company. Consult the PythonEvents page to see if any training courses are scheduled in your area and the PythonTraining page for a list of trainers.

Teachers can join the EDU-SIG, a mailing list for discussion of Python’s use in teaching at any level ranging from K-12 up to university.

Globals

These module globals must be defined:

String constant stating the supported DB API level.

Currently only the strings «1.0» and «2.0» are allowed. If not given, a DB-API 1.0 level interface should be assumed.

Integer constant stating the level of thread safety the interface supports. Possible values are:

threadsafety Meaning
Threads may not share the module.
1 Threads may share the module, but not connections.
2 Threads may share the module and connections.
3 Threads may share the module, connections and cursors.

Sharing in the above context means that two threads may use a resource without wrapping it using a mutex semaphore to implement resource locking. Note that you cannot always make external resources thread safe by managing access using a mutex: the resource may rely on global variables or other external sources that are beyond your control.

See Also

  • DocXmlRpcServer — self-documenting XML-RPC servers

  • CgiScripts — using invoked CGI scripts, rather than running micro-web servers

Discussion

I’d ultimately like to see a BaseHttpServer here that can both handle XML-RPC requests (with that request handler,) and normal web requests (with a custom handler.)

Yes- I know and love TwistedPython. But I want to make something that works in a single install. — LionKimbro 2004-05-31 01:13:16

I’d also like to add code here showing how to service a POST request.

— LionKimbro 2004-07-03 23:07:53

There exist tools like CherryPy which will create a single-file Python HTTP server (based on BaseHTTPServer). This is a fair amount easier to work with than the raw BaseHTTPServer. For most cases, using a more complete framework will be preferable (see WebProgramming). — IanBicking

I like the BaseHttpServer because it is in the default Python distributions. I encourage all work towards putting a standard web framework into the default Python distribution. I’m not picky, just as long as something is chosen. — LionKimbro 2005-01-25 04:53:53


What’s the matter with server_close()? I can call the method, but it is undocumented (see http://docs.python.org/lib/node634.html). Could someone knowledgable either remove the calls, or add a comment why they’re necessary? Thanks. — Anonymous Coward, 23 Oct 2007

http.client¶

См.также

  • https://docs.python.org/2/library/httplib.html
  • https://docs.python.org/3/library/http.client.html

представляет собой простую обертку вокруг модуля , которая обеспечивает наибольший контроль при обращении к web-сайту.

Отправка запроса.

import http.client
conn = http.client.HTTPConnection("lectureswww.readthedocs.org")
conn.request("GET", "/ru/latest/")
r1 = conn.getresponse()
print(r1.status)

data1 = r1.read()
conn.request("GET", "/parrot.spam")
r2 = conn.getresponse()
print(r2.status)

data2 = r2.read()
conn.close()
200
404

В переменных , хранится тело ответа.

запрос, с использованием модуля для преобразования Python словаря в строку параметров для HTTP запроса:

import http.client
import urllib.parse

params = urllib.parse.urlencode(
    {'@number' 12524, '@type' 'issue', '@action' 'show'}
)
headers = {"Content-type" "application/x-www-form-urlencoded",
           "Accept" "text/plain"}
conn = http.client.HTTPConnection("bugs.python.org")
conn.request("POST", "", params, headers)
response = conn.getresponse()
print(response.status, response.reason)

data = response.read()
print(data)

conn.close()

Библиотека Requests: HTTP for Humans

При решении различных задач в сфере веб-разработки, нам часто приходится взаимодействовать с HTTP. Это не самая простая задача для любого языка программирования и Python в этом не исключение. Язык, конечно, содержит встроенные модули, позволяющие отправлять , но, как это ни парадоксально, их использование едва ли можно отнести к Pythonic-way.

В своё время чтобы обойти монструозность и сложность использования встроенных модулей появилась . На данный момент она является одной из самых популярных библиотек на github: более 40 000 «звёзд» и используется более, чем в 20 000 open-source проектах. Нередко она используется и в коммерческой разработке.

Parameter Values

Parameter Description
url Try it Required. The url of the request
params Try it Optional. A dictionary, list of tuples or bytes to send as a query string.Default
allow_redirects Try it Optional. A Boolean to enable/disable redirection.Default (allowing redirects)
auth Try it Optional. A tuple to enable a certain HTTP authentication.Default
cert Try it Optional. A String or Tuple specifying a cert file or key.Default
cookies Try it Optional. A dictionary of cookies to send to the specified url.Default
headers Try it Optional. A dictionary of HTTP headers to send to the specified url.Default
proxies Try it Optional. A dictionary of the protocol to the proxy url.Default
stream Try it Optional. A Boolean indication if the response should be immediately downloaded (False) or streamed (True).Default
timeout Try it Optional. A number, or a tuple, indicating how many seconds to wait for the client to make a connection and/or send a response.Default which means the request will continue until the connection is closed
verify Try it Try it Optional. A Boolean or a String indication to verify the servers TLS certificate or not.Default

Server Extension APIs

Some server authors may wish to expose more advanced APIs, that application or framework authors can use for specialized purposes. For example, a gateway based on mod_python might wish to expose part of the Apache API as a WSGI extension.

In the simplest case, this requires nothing more than defining an environ variable, such as mod_python.some_api. But, in many cases, the possible presence of middleware can make this difficult. For example, an API that offers access to the same HTTP headers that are found in environ variables, might return different data if environ has been modified by middleware.

In general, any extension API that duplicates, supplants, or bypasses some portion of WSGI functionality runs the risk of being incompatible with middleware components. Server/gateway developers should not assume that nobody will use middleware, because some framework developers specifically intend to organize or reorganize their frameworks to function almost entirely as middleware of various kinds.

So, to provide maximum compatibility, servers and gateways that provide extension APIs that replace some WSGI functionality, must design those APIs so that they are invoked using the portion of the API that they replace. For example, an extension API to access HTTP request headers must require the application to pass in its current environ, so that the server/gateway may verify that HTTP headers accessible via the API have not been altered by middleware. If the extension API cannot guarantee that it will always agree with environ about the contents of HTTP headers, it must refuse service to the application, e.g. by raising an error, returning None instead of a header collection, or whatever is appropriate to the API.

Similarly, if an extension API provides an alternate means of writing response data or headers, it should require the start_response callable to be passed in, before the application can obtain the extended service. If the object passed in is not the same one that the server/gateway originally supplied to the application, it cannot guarantee correct operation and must refuse to provide the extended service to the application.

These guidelines also apply to middleware that adds information such as parsed cookies, form variables, sessions, and the like to environ. Specifically, such middleware should provide these features as functions which operate on environ, rather than simply stuffing values into environ. This helps ensure that information is calculated from environ after any middleware has done any URL rewrites or other environ modifications.

Application Configuration

This specification does not define how a server selects or obtains an application to invoke. These and other configuration options are highly server-specific matters. It is expected that server/gateway authors will document how to configure the server to execute a particular application object, and with what options (such as threading options).

Framework authors, on the other hand, should document how to create an application object that wraps their framework’s functionality. The user, who has chosen both the server and the application framework, must connect the two together. However, since both the framework and the server now have a common interface, this should be merely a mechanical matter, rather than a significant engineering effort for each new server/framework pair.

Finally, some applications, frameworks, and middleware may wish to use the environ dictionary to receive simple string configuration options. Servers and gateways should support this by allowing an application’s deployer to specify name-value pairs to be placed in environ. In the simplest case, this support can consist merely of copying all operating system-supplied environment variables from os.environ into the environ dictionary, since the deployer in principle can configure these externally to the server, or in the CGI case they may be able to be set via the server’s configuration files.

Applications should try to keep such required variables to a minimum, since not all servers will support easy configuration of them. Of course, even in the worst case, persons deploying an application can create a script to supply the necessary configuration values:

from the_app import application

def new_app(environ, start_response):
    environ = 'something'
    return application(environ, start_response)

HTTP 1.1 Expect/Continue

Servers and gateways that implement HTTP 1.1 must provide transparent support for HTTP 1.1’s «expect/continue» mechanism. This may be done in any of several ways:

  1. Respond to requests containing an Expect: 100-continue request with an immediate «100 Continue» response, and proceed normally.
  2. Proceed with the request normally, but provide the application with a wsgi.input stream that will send the «100 Continue» response if/when the application first attempts to read from the input stream. The read request must then remain blocked until the client responds.
  3. Wait until the client decides that the server does not support expect/continue, and sends the request body on its own. (This is suboptimal, and is not recommended.)

Задания для самоподготовки

1. Пользователь вводит произвольные целые числа и нужно создать словарь, у которого ключами будут только четные числа, а значениями – квадраты этих чисел.

2. Пусть имеется вот такая строка:

«int= целое число, dict=словарь, list=список, str=строка, bool=булевый тип»

Требуется из нее создать словарь с ключами:

int, dict, list, str, bool

и соответствующими значениями.

3. Пользователь вводит с клавиатуры M раз данные в формате:

английское слово: перевод1, перевод2, …, переводN

каждую введенную строку необходимо преобразовать и поместить в словарь, у которого ключом будет английское слово, а значением список:

Видео по теме

Python 3 #1: установка и запуск интерпретатора языка

Python 3 #2: переменные, оператор присваивания, типы данных

Python 3 #3: функции input и print ввода/вывода

Python 3 #4: арифметические операторы: сложение, вычитание, умножение, деление, степень

Python 3 #5: условный оператор if, составные условия с and, or, not

Python 3 #6: операторы циклов while и for, операторы break и continue

Python 3 #7: строки — сравнения, срезы строк, базовые функции str, len, ord, in


Python 3 #8: методы строк — upper, split, join, find, strip, isalpha, isdigit и другие

Python 3 #9: списки list и функции len, min, max, sum, sorted

Python 3 #10: списки — срезы и методы: append, insert, pop, sort, index, count, reverse, clear

Python 3 #11: списки — инструмент list comprehensions, сортировка методом выбора

Python 3 #12: словарь, методы словарей: len, clear, get, setdefault, pop

Python 3 #13: кортежи (tuple) и операции с ними: len, del, count, index

Python 3 #14: функции (def) — объявление и вызов

Python 3 #15: делаем «Сапер», проектирование программ «сверху-вниз»

Python 3 #16: рекурсивные и лямбда-функции, функции с произвольным числом аргументов

Python 3 #17: алгоритм Евклида, принцип тестирования программ

Python 3 #18: области видимости переменных — global, nonlocal

Python 3 #19: множества (set) и операции над ними: вычитание, пересечение, объединение, сравнение

Python 3 #20: итераторы, выражения-генераторы, функции-генераторы, оператор yield

Python 3 #21: функции map, filter, zip

Python 3 #22: сортировка sort() и sorted(), сортировка по ключам


Python 3 #23: обработка исключений: try, except, finally, else

Python 3 #24: файлы — чтение и запись: open, read, write, seek, readline, dump, load, pickle

Python 3 #25: форматирование строк: метод format и F-строки

Python 3 #26: создание и импорт модулей — import, from, as, dir, reload

Python 3 #27: пакеты (package) — создание, импорт, установка (менеджер pip)

Python 3 #28: декораторы функций и замыкания

Python 3 #29: установка и порядок работы в PyCharm

Buffering and Streaming

Generally speaking, applications will achieve the best throughput by buffering their (modestly-sized) output and sending it all at once. This is a common approach in existing frameworks such as Zope: the output is buffered in a StringIO or similar object, then transmitted all at once, along with the response headers.

The corresponding approach in WSGI is for the application to simply return a single-element iterable (such as a list) containing the response body as a single string. This is the recommended approach for the vast majority of application functions, that render HTML pages whose text easily fits in memory.

For large files, however, or for specialized uses of HTTP streaming (such as multipart «server push»), an application may need to provide output in smaller blocks (e.g. to avoid loading a large file into memory). It’s also sometimes the case that part of a response may be time-consuming to produce, but it would be useful to send ahead the portion of the response that precedes it.

In these cases, applications will usually return an iterator (often a generator-iterator) that produces the output in a block-by-block fashion. These blocks may be broken to coincide with multipart boundaries (for «server push»), or just before time-consuming tasks (such as reading another block of an on-disk file).

WSGI servers, gateways, and middleware must not delay the transmission of any block; they must either fully transmit the block to the client, or guarantee that they will continue transmission even while the application is producing its next block. A server/gateway or middleware may provide this guarantee in one of three ways:

  1. Send the entire block to the operating system (and request that any O/S buffers be flushed) before returning control to the application, OR
  2. Use a different thread to ensure that the block continues to be transmitted while the application produces the next block.
  3. (Middleware only) send the entire block to its parent gateway/server

By providing this guarantee, WSGI allows applications to ensure that transmission will not become stalled at an arbitrary point in their output data. This is critical for proper functioning of e.g. multipart «server push» streaming, where data between multipart boundaries should be transmitted in full to the client.

In order to better support asynchronous applications and servers, middleware components must not block iteration waiting for multiple values from an application iterable. If the middleware needs to accumulate more data from the application before it can produce any output, it must yield an empty string.

To put this requirement another way, a middleware component must yield at least one value each time its underlying application yields a value. If the middleware cannot yield any other value, it must yield an empty string.

This requirement ensures that asynchronous applications and servers can conspire to reduce the number of threads that are required to run a given number of application instances simultaneously.

Note also that this requirement means that middleware must return an iterable as soon as its underlying application returns an iterable. It is also forbidden for middleware to use the write() callable to transmit data that is yielded by an underlying application. Middleware may only use their parent server’s write() callable to transmit data that the underlying application sent using a middleware-provided write() callable.

Supporting Older (

Some servers, gateways, or applications may wish to support older (<2.2) versions of Python. This is especially important if Jython is a target platform, since as of this writing a production-ready version of Jython 2.2 is not yet available.

For servers and gateways, this is relatively straightforward: servers and gateways targeting pre-2.2 versions of Python must simply restrict themselves to using only a standard «for» loop to iterate over any iterable returned by an application. This is the only way to ensure source-level compatibility with both the pre-2.2 iterator protocol (discussed further below) and «today’s» iterator protocol (see PEP 234).

(Note that this technique necessarily applies only to servers, gateways, or middleware that are written in Python. Discussion of how to use iterator protocol(s) correctly from other languages is outside the scope of this PEP.)

For applications, supporting pre-2.2 versions of Python is slightly more complex:

  • You may not return a file object and expect it to work as an iterable, since before Python 2.2, files were not iterable. (In general, you shouldn’t do this anyway, because it will perform quite poorly most of the time!) Use wsgi.file_wrapper or an application-specific file wrapper class. (See for more on wsgi.file_wrapper, and an example class you can use to wrap a file as an iterable.)
  • If you return a custom iterable, it must implement the pre-2.2 iterator protocol. That is, provide a __getitem__ method that accepts an integer key, and raises IndexError when exhausted. (Note that built-in sequence types are also acceptable, since they also implement this protocol.)

Finally, middleware that wishes to support pre-2.2 versions of Python, and iterates over application return values or itself returns an iterable (or both), must follow the appropriate recommendations above.

Производительность

Ниже приведен фрагмент HTTP-клиента, отправляющего запросы на httpbin.org, HTTP-API, который обеспечивает (среди прочего) конечную точку, имитирующую длинный запрос. Этот пример реализует все методы, перечисленные выше.

Программа для сравнения производительности использования различных запросов

import contextlib
import time

import aiohttp
import asyncio
import requests
from requests_futures import sessions

URL = "http://httpbin.org/delay/1"
TRIES = 10


@contextlib.contextmanager
def report_time(test):
    t0 = time.time()
    yield
    print("Time needed for `%s' called: %.2fs"
          % (test, time.time() - t0))


with report_time("serialized"):
    for i in range(TRIES):
        requests.get(URL)


session = requests.Session()
with report_time("Session"):
    for i in range(TRIES):
        session.get(URL)


session = sessions.FuturesSession(max_workers=2)
with report_time("FuturesSession w/ 2 workers"):
    futures = 
    for f in futures:
        f.result()


session = sessions.FuturesSession(max_workers=TRIES)
with report_time("FuturesSession w/ max workers"):
    futures = 
    for f in futures:
        f.result()


async def get(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            await response.read()

loop = asyncio.get_event_loop()
with report_time("aiohttp"):
    loop.run_until_complete(
        asyncio.gather(*))

Запуск этой программы дает следующий вывод:

Time needed for `serialized' called: 12.12s
Time needed for `Session' called: 11.22s
Time needed for `FuturesSession w/ 2 workers' called: 5.65s
Time needed for `FuturesSession w/ max workers' called: 1.25s
Time needed for `aiohttp' called: 1.19s

Не удивительно, что более медленный результат приходит с сериализованной версией, поскольку все запросы выполняются один за другим без повторного использования соединения — 12 секунд на 10 запросов.

Использование объекта Session и, следовательно, повторное использование соединения означает экономию 8% времени, что уже является большим и легким выигрышем. Как минимум, вы всегда должны использовать Session.

Если ваша система и программа допускают использование потоков, рекомендуется использовать их для распараллеливания запросов. Однако у потоков есть некоторые накладные расходы, и они не менее весовые. Они должны быть созданы, запущены и затем присоединены.

Если вы не используете старые версии Python, то, без сомнения, использование aiohttp должно быть вашим выбором, если вы хотите написать быстрый и асинхронный HTTP-клиент. Это самое быстрое и масштабируемое решение, поскольку оно может обрабатывать сотни параллельных запросов.


С этим читают