Эффективные способы параллельного вызова API в Python

Для параллельного вызова API в Python вы можете использовать различные методы в зависимости от ваших требований. Вот несколько популярных подходов с примерами кода:

  1. Использование модуля concurrent.futures:

    import concurrent.futures
    import requests
    # Define a function to make API calls
    def make_api_call(url):
    response = requests.get(url)
    return response.json()
    # List of API URLs
    api_urls = ["https://api1.example.com", "https://api2.example.com", "https://api3.example.com"]
    # Create a thread pool executor
    with concurrent.futures.ThreadPoolExecutor() as executor:
    # Submit API calls to the executor
    results = [executor.submit(make_api_call, url) for url in api_urls]
    # Obtain the results as they become available
    for future in concurrent.futures.as_completed(results):
        response = future.result()
        # Process the API response as needed
        print(response)
  2. Использование модуля multiprocessing:

    import multiprocessing
    import requests
    # Define a function to make API calls
    def make_api_call(url):
    response = requests.get(url)
    return response.json()
    # List of API URLs
    api_urls = ["https://api1.example.com", "https://api2.example.com", "https://api3.example.com"]
    # Create a process pool
    with multiprocessing.Pool() as pool:
    # Map the API call function to the URLs
    results = pool.map(make_api_call, api_urls)
    # Process the results
    for response in results:
        # Process the API response as needed
        print(response)
  3. Использование модуля asyncioс aiohttp:

    import asyncio
    import aiohttp
    # Define a coroutine to make API calls
    async def make_api_call(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.json()
    # List of API URLs
    api_urls = ["https://api1.example.com", "https://api2.example.com", "https://api3.example.com"]
    # Create an event loop
    loop = asyncio.get_event_loop()
    # Gather the API call coroutines
    coroutines = [make_api_call(url) for url in api_urls]
    # Run the event loop to execute the coroutines concurrently
    results = loop.run_until_complete(asyncio.gather(*coroutines))
    # Process the results
    for response in results:
    # Process the API response as needed
    print(response)

Эти методы позволяют выполнять вызовы API параллельно, что может значительно повысить производительность при работе с несколькими конечными точками API.