The reliable way to block a bad bot on Apache is to reject its request server-side, usually with Apache 2.4 access-control rules or mod_rewrite. A robots.txt file can ask cooperative crawlers to stay away, but it cannot stop a scraper that ignores the request. Because User-Agent headers are supplied by the client and are easy to change, treat a User-Agent denylist as a coarse filter—not as authentication or a complete anti-attack system.
Antihost offers Linux VPS plans with full root access, unlimited traffic on a 1 Gbps port and a 14-day free trial.
Start with your access logs, identify the smallest path and signature you need to block, apply one of the configurations below, and confirm that legitimate players, search crawlers, APIs, and game clients still work. If requests continue at a damaging rate, move the control upstream to a firewall, managed WAF, or CDN security layer.
As an Amazon Associate I earn from qualifying purchases.
Choose the right layer before changing Apache
Bot controls differ in what they can enforce and when they act. The choice matters for a game site because a crawler may hit public news pages, a leaderboard endpoint, a login form, or a download directory at very different rates.
| Control | What it does | Resistance to evasion | Typical scope | Where rejection happens |
|---|---|---|---|---|
robots.txt |
Publishes a voluntary crawler-exclusion instruction | Weak; a hostile client can ignore it | Site paths described in the file | Nowhere on the server; the client chooses whether to comply |
| Apache User-Agent rule | Returns an Apache denial such as HTTP 403 for matching requests | Weak against clients that change headers; useful against self-identifying or low-effort bots | A directory, URL path, virtual host, or selected requests, depending on rule placement | Apache, after the request reaches the web server |
| Firewall, managed WAF, or CDN security layer | Filters traffic before it reaches the origin, using controls beyond a User-Agent string | Stronger than header-only matching, depending on the service and policy | Host, application, network, or rule-specific scope | Upstream of Apache |
Apache’s documentation recommends considering a higher-level block when an attack is sustained. An origin rule can still be useful for a narrow, known signature, but it does not remove the bandwidth or connection cost of traffic that has already reached Apache.
#1 Best Overall
Inspect logs and define a narrow target
Do not begin with a broad pattern such as “block every crawler.” A User-Agent string is untrusted input, and a loose match can deny a legitimate player, accessibility tool, search crawler, or monitoring service. First identify the requests that are actually causing a problem.
Record the evidence you need
- The exact User-Agent spelling and capitalization variants.
- The requested path, including whether the traffic is limited to an API, download directory, admin area, or other resource.
- The source address or address range when the same actor consistently uses one, while remembering that addresses can change or be shared.
- The response status and request rate shown in your access logs.
- Whether legitimate traffic uses the same path or a similar User-Agent.
Look for repeated patterns over a useful time window rather than blocking after one unusual request. Keep a record of the date, pattern, rule location, and reason for each denylist entry so it can be removed when the traffic changes.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Decide whether a path or the whole site needs protection
Scope a rule to the smallest practical path. A bot scraping a public leaderboard does not automatically justify blocking it from every page on the virtual host. Narrow scope reduces accidental outages and makes rollback easier. Use a host-wide rule only when the evidence shows that the same unwanted client is harmful everywhere on that host.
Option 1: deny a User-Agent with Apache 2.4 access expressions
Apache 2.4 supports an expression-based access rule. This is the shortest configuration when you need to deny a known string and your server configuration context allows the <If> container.
<If "%{HTTP_USER_AGENT} == 'BadBot'">
Require all denied
</If>Replace BadBot with the exact value you observed. The expression shown is case-sensitive. If the unwanted client varies capitalization, use a case-insensitive expression or choose the SetEnvIfNoCase method below, which is designed for case-insensitive matching.
Limit the expression to a protected area
Place the rule in the narrowest configuration context that covers the resource you need to protect. A directory-level rule is safer than a host-wide rule when only one game endpoint is being abused. If you are editing .htaccess, your Apache administrator must have enabled the required override class; otherwise Apache will reject the file or ignore the intended control. In a virtual-host or server configuration, reload Apache using your normal, distribution-specific procedure after validating the syntax.
Rank #2
- Used Book in Good Condition
What result to expect
A matching request should receive an HTTP 403 Forbidden response. The request may still appear in Apache’s access log because Apache had to receive and evaluate it. That is different from an upstream firewall drop, which can prevent the connection from reaching the origin at all.
Option 2: use SetEnvIfNoCase and Require
Apache’s documented access-control recipe sets an environment flag when the User-Agent matches, then denies that flag only inside the protected location:
SetEnvIfNoCase User-Agent ^NameOfBadRobot goaway
<Location "/secret/files">
<RequireAll>
Require all granted
Require not env goaway
</RequireAll>
</Location>SetEnvIfNoCase makes the match case-insensitive, so NameOfBadRobot also catches capitalization changes in that string. The caret (^) anchors the pattern at the start of the header. Remove the anchor only when your evidence shows the identifying text can appear later in the User-Agent; a broader pattern also increases the chance of a false positive.
Understand the configuration context
The <Location> container belongs in server or virtual-host configuration, not in an ordinary .htaccess file. If you only control .htaccess, use a directory-appropriate access rule or a mod_rewrite rule instead, subject to the override permissions set by the host administrator. Do not paste a <Location> block into a context that does not permit it.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Protect one endpoint without affecting the rest of the site
Change /secret/files to the real path you need to protect, such as a download or API directory. Keep the path specific. If a bot only targets /leaderboard/export, denying every request on the host can disrupt unrelated pages and game services.
Option 3: return 403 with mod_rewrite
mod_rewrite can inspect %{HTTP_USER_AGENT} and stop a matching request with the [F] flag. The flag produces a 403 response. This is useful when you need path matching, several conditions, or a combination of User-Agent and IP evidence.
RewriteEngine On
RewriteCond "%{HTTP_USER_AGENT}" "^NameOfBadRobot"
RewriteCond "%{REMOTE_ADDR}" "=123.45.67.[8-9]"
RewriteRule "^/secret/files/" "-" [F]This example denies requests only when all three conditions are true: the User-Agent begins with the selected string, the remote address matches the shown range pattern, and the URL is under /secret/files/. Replace every example value with evidence from your logs. The IP condition is optional; include it only when the address is a meaningful part of the observed pattern.
Match several known names carefully
You can combine alternatives in one regular expression, but every added alternative broadens the denylist. For example, a pattern such as ^(NameOfBadRobot|AnotherBadRobot) matches either leading string. Keep the list short, comment why each entry exists, and remove entries that no longer describe harmful traffic.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsUse the URL path deliberately
The rule’s path must match the URL form available in the configuration context. A rule in a server or virtual-host context commonly sees a leading slash; directory-level rules can have a path prefix removed before matching. If a rule appears correct but never fires, verify the context and the path representation in Apache’s rewrite documentation or your host’s configuration guidance rather than widening the expression blindly.
Why User-Agent blocking is only a coarse filter
The Apache HTTP Server Project explicitly warns that any technique relying on the USER_AGENT string can be trivially circumvented because the client can change that string. A determined scraper can copy a browser header, rotate names, or omit the header entirely. Therefore:
- Do not use a User-Agent denylist to authenticate a legitimate search crawler or game client.
- Do not treat a matching header as proof that two requests came from the same actor.
- Do not assume a 403 rule eliminates the attacker’s traffic; it only rejects requests that reach Apache and still match the rule.
- Use the denylist for self-identifying or low-effort bots, noisy automation, and a quick reduction in unwanted requests.
If the attacker changes headers and continues, collect the new indicators from logs and consider controls that do not depend on one client-supplied field. An upstream firewall or managed WAF can apply additional policy before the origin has to process the request.
robots.txt: useful signal, not enforcement
A robots.txt file is a voluntary crawler-exclusion mechanism. It tells cooperative crawlers which paths they should not request, but it does not create an Apache denial and does not authenticate the clients that comply. A hostile bot can read the file, ignore it, or use the listed paths as a map of interesting content.
Free tools Windows power users keep installed
One-click scans. No signup required.
Use robots.txt when your goal is to communicate preferences to well-behaved crawlers. Add an Apache rule, application control, or upstream security policy when the goal is to stop requests. Do not claim that a robots.txt entry has blocked traffic unless a separate enforcement layer does so.
Validate a rule without locking out legitimate users
- Back up the active configuration. Save the current virtual-host file or
.htaccessbefore editing so you can restore it quickly. - Check the directive context. Confirm that your hosting plan permits the directive and that the module required by the chosen method is enabled.
- Validate syntax before reload. Use the Apache configuration test command supplied by your installation, then reload only after it reports a valid configuration.
- Test a matching request. Send a request to the protected path with the exact User-Agent pattern and confirm that Apache returns 403.
- Test a non-matching request. Use a normal browser or a clearly different User-Agent and confirm that an allowed resource still returns its expected response.
- Test the boundary. Request a nearby path outside the protected scope to ensure the rule is not broader than intended.
- Review access and error logs. Confirm which rule fired, whether legitimate clients were denied, and whether the traffic volume changed.
When testing from a shared network, remember that the source address may represent many players. An IP condition that looks precise in one log sample can affect unrelated users later.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Troubleshoot common failures
The server returns a 500 error after editing
A syntax error, unsupported directive, or disallowed .htaccess override commonly causes this symptom. Restore the backup, run the configuration test, and check Apache’s error log for the directive and line number. Do not keep adding rules until the configuration validates.
The bot is still receiving a normal response
Check capitalization, leading text, and whitespace in the actual header. Confirm that the request reaches the virtual host where the rule is loaded, and verify that the URL path matches the rule’s context. If the client changed its User-Agent, the old denylist may be working exactly as written while no longer matching the new traffic.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Legitimate players receive 403
Remove or narrow the broad pattern, then identify which legitimate client shares the matched text. Scope the rule to the abused path, replace a generic substring with a more specific signature, or remove the User-Agent condition if it cannot distinguish the traffic safely. Never use a denylist as a substitute for player authentication.
The rule works on one URL but not another
Compare the path, trailing slash, directory context, and virtual host handling. A rule scoped to /secret/files/ does not automatically cover a similarly named path elsewhere. Keep separate rules explicit rather than silently expanding one pattern.
Best Value
- Used Book in Good Condition
Apache is overloaded even though 403 responses are appearing
A 403 generated by Apache still means the request reached the origin and consumed some processing. If the request rate, connection count, bandwidth, or upstream resource use remains harmful, move mitigation to a higher layer. The Apache documentation specifically recommends considering firewall-level blocking for sustained attacks.
When to escalate to a firewall, WAF, or CDN security layer
Escalate when the problem is sustained, distributed, expensive to serve, or able to evade header matching. Upstream controls are appropriate when you need to reduce traffic before Apache, protect several origins, or combine signals that Apache alone does not evaluate conveniently.
Recommended Free Tools
Signals that an origin denylist is no longer enough
- Requests continue after the User-Agent is changed or removed.
- Many addresses generate the same traffic pattern.
- Apache workers, bandwidth, database connections, or API quotas are being exhausted.
- The unwanted traffic targets several paths or virtual hosts.
- You need rate limits, challenge actions, reputation data, or network-level drops rather than a simple 403.
The exact product, pricing, geography, and policy options depend on the provider and your deployment. Verify those details separately before selecting a service. The general decision is architectural: reject earlier when origin-level rules cannot keep the site responsive.
Maintain the denylist as an operational control
Document every rule
Comment the reason, date, affected path, and evidence for each pattern. A future administrator should be able to tell whether an entry targets a known scraper, a temporary incident, or a test. Include a removal condition when possible, such as deleting a temporary rule after traffic normalizes.
Review for false positives
Compare denied requests with support reports, analytics, game-client logs, and crawler verification processes. Because clients can forge User-Agent values, a name that sounds official is not proof of identity. Conversely, an unfamiliar name is not automatically malicious.
Keep scope and matching style consistent
Use case-insensitive matching when spelling varies, but avoid unnecessary wildcards. Keep a protected path narrow, and prefer separate, explainable entries over one opaque expression that is difficult to audit. If the same actor moves across paths or addresses, treat that as a reason to reassess the enforcement layer rather than continually widening Apache regexes.
A practical decision path
- Only cooperative crawlers need guidance: publish an appropriate
robots.txtpolicy. - A known, self-identifying bot is wasting requests on one path: use an Apache 2.4 expression,
SetEnvIfNoCase, or a path-scopedmod_rewrite403 rule. - You need to combine a header with a known address range: use conditions in
mod_rewrite, while recognizing that both values can change. - The traffic is sustained or costly: ask for a firewall, managed WAF, or CDN security control that can act before Apache.
For most Apache game sites, the safest first change is a narrow, logged 403 rule for a verified signature, followed by monitoring. That approach addresses the immediate nuisance without pretending that a client-controlled header can stop a determined attacker.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
