chore: refactor concurrency.py to run sync context managers in a single thread#5707
chore: refactor concurrency.py to run sync context managers in a single thread#5707adriangb wants to merge 14 commits into
Conversation
|
📝 Docs preview for commit e24e016 at: https://63851d6f5737095d3317664e--fastapi.netlify.app |
|
This means more resource optimised DB connections?🤔 (just wondering) |
|
Not really. It just means that sync context manager's |
|
Make sense, better house-keeping! Thanks for your explanation btw! 😅 |
|
📝 Docs preview for commit e45a801 at: https://63852343acf33a6f2fa9c78e--fastapi.netlify.app |
|
marked as ready, cc @tiangolo |
| await anyio.to_thread.run_sync( | ||
| cm.__exit__, None, None, None, limiter=exit_limiter | ||
| cm: ContextManager[T], | ||
| limiter: Optional[anyio.CapacityLimiter] = None, |
There was a problem hiding this comment.
This is currently unused but IMO exposing it makes sense for future customizability. Happy to remove it if not
|
📝 Docs preview for commit ac93694 at: https://638663ca7c5147091cf84317--fastapi.netlify.app |
|
📝 Docs preview for commit 6ba173b at: https://638a9f62bb6c0b319b2736fb--fastapi.netlify.app |
|
The Issue here |
|
Awesome! Did you get to test this branch? |
Yeah, it didn't have any effect. |
|
📝 Docs preview for commit 5ac3f62 at: https://63a1f7af8c940416a38c635d--fastapi.netlify.app |
|
📝 Docs preview for commit 04347ed at: https://63a4d7eaeccde823d1092dc0--fastapi.netlify.app |
|
Is this pr possible to merge sometime soon? Coming here from this thread: MagicStack/asyncpg#978 |
|
For visibility: #8424 (comment) |
|
The test from this PR doesn't fail with current FastAPI version (0.116.0): import threading
from contextvars import ContextVar
from fastapi import Depends, FastAPI
from fastapi.testclient import TestClient
ctx: ContextVar[str] = ContextVar("ctx")
async def context_same_thread():
# test both a real world use case and a
# simpler synthetic check
ctx.set("foo")
ident = threading.get_ident()
yield
assert ctx.get() == "foo"
assert ident == threading.get_ident()
app = FastAPI()
@app.get("/check_cm_same_thread")
async def check_cm_same_thread(_=Depends(context_same_thread)):
return None
def test_sync_runs_in_single_thread():
client = TestClient(app)
response = client.get("/check_cm_same_thread")
response.raise_for_status()@adriangb, could you please double-check? Seems that it has been fixed. UPD: shouldn't |
|
As this PR has been waiting for the original user for a while but seems to be inactive, it's now going to be closed. But if there's anyone interested, feel free to create a new PR. |
|
It's unfair to expect users to reply when review takes so long. Isn't there a less aggressive strategy? Like pinging the user again a couple of weeks? |
@Kludex, I agree we should change this. What should we do with this PR now? |
|
This pull request has a merge conflict that needs to be resolved. |
|
I'm realizing that this was related to a change done a while ago with dependencies with yield, this was solved in 0.74.0 (https://fastapi.tiangolo.com/release-notes/#0740) The exit code after This was because custom middlewares would internally create an AnyIO task group, which creates a new context. That logic was then refactored once again in 0.106.0 (https://fastapi.tiangolo.com/release-notes/#01060). And is now refactored again here 😅 : #14099 , which will be released in version 0.118.0 But the most important point is, a test for this use case was added (in the first PR in the version described above) here: https://github.com/fastapi/fastapi/blob/master/tests/test_dependency_contextvars.py So that should ensure this is handled/fixed and stays handled and fixed. 🚀 Given that, I'll now close this one, but if anyone faces a similar problem again, please create a new discussion with an example to replicate it so we can take a look. 🤓 Thanks all! 🍰 |
I'm hoping that this (maybe) helps resolve issues like MagicStack/asyncpg#978
This change makes sync context managers acquire and hold onto a thread for the entirety of their execution so that we don't split
__enter__and__exit__calls across threads. I think this will resolve a whole class of intermittent bugs that might crop up for anything using thread locals or context variables.The main "con" of this approach is that we won't "free" the thread in between
__enter__and__exit__which means more contention for threads in the thread pool. In other words, despite the 40 thread default limit that anyio imposes it was previously possible to run more than 40 sync context managers as FastAPI dependencies since the 41st would just have to wait for any__enter__to finish before it could acquire a thread pool. With this change running more than 40 sync context managers will result in a deadlock. I see three options: