In this video, we learn how to massively speed up web scraping with asynchronous requests in Python, using the httpx module.
◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
📚 Programming Books & Merch 📚
🐍 The Python Bible Book:
💻 The Algorithm Bible Book:
👕 Programming Merch:
🌐 Social Media & Contact 🌐
📱 Website:
📷 Instagram:
🐦 Twitter:
🤵 LinkedIn:
📁 GitHub:
🎙 Discord:
🎵 Outro Music From:
Hi ,
I tried executing HTTPX POST method, but it is returning 419 instead of 200. the calls are similar to what I used to do with request.Session.. can you please create a view on POST call. what is the trick to make it work ?
Very cool.
What's your take on concurrent.futures?
“`
import concurrent.futures
import time
import requests
start = time.monotonic()
with concurrent.futures.ThreadPoolExecutor() as executor:
executor.map(lambda u: print(requests.get(u).status_code), urls)
end = time.monotonic()
print(f'{end – start:.2f}s')
“`
Comment for the algorithm 🙂 Perfect video lesson
This is INSANE!
very interesting, it is possible to also have the url that we are testing with the return 200
wow, it works amazingly. Thanks for sharing this
Big ups for all the stuff you do man! All videos are very clear, concise and on point. Keep it up
Can you solve rubik cube with python if yes then please make video
Ich habe mit einem 2 Jahre alten Video von dir gestartet – und hatte gehofft dass du immer noch postest … klasse das du es tust!
I’m happy you still posting! Thanks for what you do.
You're awesome dude, loving your new pace. Keep up the great work.
Is this efficient on getting access to different ip camera addresses
Thanks a lot for the video! Does "httpx" work with webdriver from selenium?
Great video!
This was a good one, really interesting.
Have you covered python queue.Queue() with threading?
Python 3.11 + httpx = 🔥🔥🔥🔥
this video was very helpful! exactly what I needed 🙂 keep it up! (this vid was suggested to me at random lol)
are the async requests that we gather in the same order as the URLS list??
Nice! I have never worked with HTTPX, so this is really interesting to me. Would love to see more!
Love from India. 4th comment.. daily viewer .. python lover ❤️🫂
Awesome video! We talked about a similar topic on our channel a few videos ago!
Thanks!
❤ just learned new thing !!