disable modsecurity for proxy requests

disable modsecurity for proxy requests

So I have this combo:
apache2, mod_ssl, a different server running on a different port and
mod_security.
I configured apache2 via ProxyPass and ProxyPassReverse to map a location
on the webserver to the "different server running on a different port".
This works, however mod_security doesn't like some action and I want to
either disable that rule or mod_security all together, but only for this
specific proxy location not side wide.
Somehow this just doesn't work:
<Location /mylocation>
<IfModule mod_security2.c>
SecRuleRemoveById 12345
</IfModule>
ProxyPass http://ip:port/location
ProxyPassReverse http://ip:port/location
</Location>
If I move the mod_security part one level up, that is vhost wide, it does
work.
It seems to me /mylocation/someurl does get proxied but not matched or
something along those lines. Anybody any pointers how I can do this the
right way?