Use rsyslog as the collection and transport layer, Logstash as the parsing and enrichment pipeline, and Elasticsearch as the searchable store. This three-tier design lets you select only the events you need, buffer outages, protect traffic with TLS, and apply one consistent schema before indexing. For a small, lightly transformed deployment, rsyslog can write directly to Elasticsearch; add Logstash when centralized parsing, routing, enrichment, or durable buffering justifies the extra service.
Antihost offers Linux VPS plans with full root access, unlimited traffic on a 1 Gbps port and a 14-day free trial.
The reference architecture
A centralized logging deployment separates collection, transport, processing, and storage. Linux hosts or an edge collector run rsyslog. Rsyslog accepts messages from local sockets, systemd’s journal, files, and network syslog listeners. Its documented pipeline is inputs → rulesets → actions: inputs receive events, rulesets classify or transform them, and actions write locally or forward elsewhere. Queues can buffer work between stages.
As an Amazon Associate I earn from qualifying purchases.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Rsyslog forwards selected events to a Logstash input over TCP or UDP. Logstash then parses the incoming records, enriches them with fields such as host or environment, routes malformed data, and sends normalized documents through its Elasticsearch output. Elasticsearch stores and indexes those documents for search, dashboards, alerting, and retention policies.
- Collectors: rsyslog on each Linux host, or on an edge collector, receives journal, socket, file, and network-syslog data.
- Transport: rsyslog sends only the required event classes to a Logstash TCP or UDP listener. Use TLS or another reliable transport when confidentiality and delivery guarantees matter.
- Processing: Logstash applies codecs, filters, enrichment, routing, and error handling before output.
- Storage and search: Elasticsearch authenticates Logstash, authorizes its index operations, and stores the resulting documents.
The rsyslog project describes rsyslog as “a modern, high-performance logging service and the default system logger on many Linux distributions.” That makes it a practical collection point, while Logstash provides a single place to maintain parsing and enrichment logic.
#1 Best Overall
When to bypass Logstash
Rsyslog includes an omelasticsearch action that can write directly to Elasticsearch. Direct output has fewer moving parts and can be appropriate when records already have the fields you need, transformations are minimal, and your team is comfortable maintaining Elasticsearch authentication and index permissions in every collector configuration.
Keep Logstash in the path when several producers need one parsing policy, when you must enrich events with consistent metadata, when different event classes require different indexes, or when you need a central place to quarantine malformed messages. The additional service creates another failure domain and requires its own capacity, queue, certificate, and upgrade management.
| Design | Best fit | Main operational trade-off |
|---|---|---|
| Rsyslog → Elasticsearch | Already-structured events with little transformation | Fewer components, but parsing and routing logic is distributed across collectors |
| Rsyslog → Logstash → Elasticsearch | Centralized parsing, enrichment, routing, and buffering | More operational overhead, with one shared processing tier to scale and protect |
Decide what the pipeline must guarantee
Classify event types first
List the facilities, severities, hosts, programs, and network devices that matter. Decide whether authentication records, firewall events, application logs, kernel messages, and audit data need different retention or access controls. A narrow forwarding rule reduces bandwidth, index growth, and accidental exposure of irrelevant data.
Choose retention and index boundaries
Define how long each class must remain searchable and when it should move to a cheaper tier or be deleted. Keep high-volume and high-value security events in separate index patterns when they have different retention, access, or rollover requirements. Index lifecycle and shard planning belong in the design, not as a repair after the cluster fills.
Classify sensitive data
Logs can contain credentials, personal data, tokens, or security telemetry. Treat the rsyslog-to-Logstash hop and the Logstash-to-Elasticsearch hop as sensitive network connections. TLS protects confidentiality in transit; authentication and least-privilege index permissions control who can send or read data.
Rank #2
- EASY TO USE - The manager notebook is easy-to-use that help you keep track of shift notes, employees, etc.
- MONITOR YOUR DATAS - Using a project manager notebook to store all your data, you can track your comps, sales, payments, and customer behavior,consult your records whenever needed.
- HIGH QUALITY - The manager office supplies is used to high quality 100gsm pure white paper, elastic band and a back pocket for extra space. Make sure you have enough space for all manager plan
- UNIQUE DESIGN & A4 SIZE - Manager log book cover is lovely, golden spiral bound design, size of 8.2" x 10.5". Just the perfectly size to fit in your backpack, purse or laptop case. Without taking up your space and always helping you keep track of your small business
- THE PERFECT GIFT - Management logbook as gift for woman & man. Use it to improve your management efficiency, make efficient adjustments whenever needed
Set a delivery policy
UDP has low overhead but can lose datagrams during congestion, host failure, or a network interruption. Use it only when loss is acceptable. TCP maintains a stateful connection and is the usual choice when delivery matters. Add TLS when the transport crosses an untrusted or shared network, or whenever the records themselves require confidentiality.
| Transport choice | Use it when | Important consequence |
|---|---|---|
| UDP | Low overhead is more important than complete delivery | Messages can be lost without a reliable retransmission path |
| TCP | You need a stateful connection and stronger delivery behavior | Connection failures must be detected and recovered |
| TCP with TLS | Logs contain credentials, personal data, or security telemetry | Both certificate validation and connection recovery must be operated |
Lay out rsyslog configuration cleanly
The primary configuration file is /etc/rsyslog.conf. Put host-specific or service-specific rules in /etc/rsyslog.d/*.conf so that packages and administrators can manage individual snippets without editing the main file. Keep inputs, rulesets, and actions visually separate.
Inputs
Use imuxsock for the local Unix socket, imjournal for systemd journal events, and imfile for application files. Network listeners use imtcp or imudp. Load only the modules you require and bind network listeners deliberately rather than exposing an unnecessary interface.
module(load="imuxsock")
module(load="imjournal")
module(load="imtcp")
input(type="imtcp" port="6514" ruleset="to_logstash")The exact TLS directives and their order depend on the rsyslog transport configuration you select. Rsyslog’s TLS central-server guidance emphasizes directive order; keep certificate, stream-driver, and listener directives in the documented order for your installed version instead of rearranging them for convenience.
Rulesets and filters
Rulesets decide which events are forwarded. Filter by facility, severity, hostname, program, or fields parsed from a message. Send only the required classes to Logstash and preserve a local safety copy when your recovery policy calls for it.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsruleset(name="to_logstash") {
if ($syslogseverity <= 4) then {
action(
type="omfwd"
target="logstash.internal"
port="6514"
protocol="tcp"
queue.type="LinkedList"
queue.filename="to-logstash"
)
}
}This example expresses the decision, destination, and queue in one place; adapt the filter and action parameters to your rsyslog release and transport policy. Do not assume that a successful configuration parse proves that Logstash is reachable or that Elasticsearch will accept the resulting document.
Actions and queues
Actions can write local files, forward remotely, or invoke omelasticsearch. A queue between collection and forwarding prevents a temporary downstream outage from immediately blocking the collector. For production, choose queue limits, disk placement, and retry behavior deliberately, and alert before the queue consumes the filesystem.
Build the Logstash pipeline
Input: TCP or UDP
Configure one Logstash input that matches the rsyslog transport. A syslog codec can parse standard syslog framing; use a different codec when the sender emits JSON or another structured format. Keep the listening port and protocol consistent on both sides.
input {
tcp {
port => 6514
codec => syslog
}
}
filter {
mutate {
add_field => { "pipeline" => "syslog" }
}
}
output {
elasticsearch {
hosts => ["https://es01.internal:9200"]
index => "syslog-%{+YYYY.MM.dd}"
user => "${ES_USER}"
password => "${ES_PASSWORD}"
ssl_certificate_authorities => ["/etc/logstash/certs/ca.crt"]
}
}The snippet is a starting shape, not a universal drop-in configuration. Match the codec to the sender, replace the index strategy with your lifecycle policy, and store credentials outside the pipeline file where your secret-management practice requires it.
Recommended Free Tools
Parse and normalize
Normalize timestamps, host names, facilities, severities, program names, and message bodies into predictable fields. Add environment, service, or collector metadata in Logstash so every producer follows the same schema. Use conditional filters for event classes rather than applying expensive parsing to every message.
Route malformed events
Do not silently discard records that fail parsing. Add a tag or field when a parser fails, then route those events to a diagnostic index or file with a bounded retention period. Monitor that path: a sudden increase usually means a device format changed, a codec is wrong, or an upstream sender is truncating messages.
Scale the processing tier
For higher volume or maintenance windows, run a horizontally scalable Logstash group and distribute senders across its endpoints. Persistent queues can retain events across a Logstash restart, but they require disk sizing, monitoring, and a recovery plan. A queue is a buffer, not unlimited storage; it only buys time until the downstream problem is corrected.
Secure Logstash’s connection to Elasticsearch
Elastic states that “Elasticsearch clusters are secured by default (starting in 8.0).” Configure the Elasticsearch output with an account intended for ingestion, HTTPS endpoints, and a trusted certificate authority. The account should have only the privileges required to create or write the target indexes and to use the index lifecycle operations your design actually needs.
- Authentication: provide a valid Logstash user and password or the credential mechanism selected by your deployment.
- Authorization: grant index permissions narrowly; do not use a broad administrative account for ingestion.
- TLS validation: install the correct certificate authority and validate the Elasticsearch certificate name and chain.
- Secret handling: keep passwords and keys out of publicly readable configuration backups and source repositories.
- Failure behavior: treat authentication errors as a pipeline outage. Elastic documents that failed authentication halts Logstash processing.
Elastic documentation also notes that the Logstash Elasticsearch output, input, and filter plugins, along with monitoring and central management, support authentication and encryption over HTTPS. Test certificate rotation before the first production rotation date; a valid certificate today does not guarantee that a future chain change will be accepted by every collector.
Implement the deployment in a controlled sequence
- Define event classes, retention, and sensitivity. Write down which facilities, devices, and applications are in scope, how long each class is retained, and which records require encryption and restricted access.
- Install and enable rsyslog. Confirm the service starts at boot. Put host-specific rules in
/etc/rsyslog.d/and keep the main file focused on shared defaults and module loading. - Create one Logstash input. Choose TCP or UDP, select the matching codec, bind the intended interface, and restrict the listening port with network controls.
- Add a narrowly scoped forwarding rule. Forward a small, known event class first. Confirm that rsyslog accepts the configuration and that the action queue behaves as expected.
- Parse and normalize in Logstash. Add fields for host, facility, severity, timestamp, program, and environment. Tag parse failures and route them to a diagnostic path.
- Configure the Elasticsearch output. Set HTTPS, credentials, the certificate authority, index naming, and the least-privilege permissions needed by the pipeline.
- Add resilience and observability. Configure rsyslog queues, Logstash persistent queues where appropriate, disk alerts, pipeline monitoring, and Elasticsearch index and shard policies before increasing the event scope.
- Exercise recovery scenarios. Test a Logstash outage, an Elasticsearch outage, duplicate delivery, clock skew, certificate rotation, malformed messages, and index rollover in the target environment. Record what is lost, delayed, retried, or duplicated for each case.
Queues, outages, and delivery semantics
Rsyslog queues buffer work between pipeline stages; the stages model and queue behavior are described in the rsyslog documentation at https://docs.rsyslog.com/doc/concepts/log_pipeline/stages.html. Use memory queues for short interruptions when losing queued data is acceptable, and disk-assisted or disk queues when recovery requirements justify the I/O and capacity cost. Set explicit size limits and alert thresholds.
Best Value
- EASY TO USE - The inventory and sales log book are easy-to-use inventory books that help you track inventory, purchases, sales, balances, unit and total costs, and manage reorders - all in one place. Easy track your inventory for small businesses.
- MONITOR YOUR DATAS - Using a sales inventory book to store all your data, you can consult your records whenever needed. Optimize your business and generate the most benefit.
- UNIQUE DESIGN - We make sure you can tailor this inventory log book to your enterprise business needs to take full advantage of its capabilities. It will work for online, consignment, home or in-store businesses.
- HIGH QUALITY - This sales book for your business, sales book size of 5.8" x 8.5", just the perfectly size to fit in your backpack, purse or laptop case. Is used to high quality 100gsm pure white paper, elastic band and a back pocket for extra space.
- THE PERFECT GIFT - Use inventory and sales log book for your personal or samll business finances, give it to your friends, family as a gift for Birthday| Easter|Children's Day|Halloween|Thanksgiving|Christmas|Back to school and New Year's Day.
When Logstash is unavailable, a TCP sender should detect the broken connection and retry according to its action and queue settings. UDP senders cannot obtain the same delivery feedback. When Elasticsearch rejects documents, Logstash needs enough queue capacity and retry time to absorb the incident without filling the host disk. Decide in advance whether old events are dropped, retained locally, or sent to a secondary diagnostic path.
At-least-once retry behavior can create duplicates after a timeout or reconnect. Include a source identifier, original timestamp, host, and message hash when deduplication matters. Do not claim exactly-once delivery unless you have implemented and verified an idempotent design end to end.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCapacity, retention, and lifecycle planning
No single throughput or latency number applies to every message size, filter set, queue policy, and Elasticsearch topology. Measure in the target environment with representative event rates and failure scenarios rather than copying a benchmark from a different deployment.
- Measure the whole path: generator or real senders, rsyslog rules, network transport, Logstash filters, and Elasticsearch indexing.
- Watch saturation: monitor CPU, memory, queue depth, disk latency, network utilization, Logstash pipeline utilization, and Elasticsearch indexing pressure.
- Plan retention from bytes: estimate daily event volume after parsing and metadata expansion, then include replicas, index overhead, queue headroom, and recovery copies.
- Control shard growth: use rollover and lifecycle policies that match event volume instead of creating a new tiny index for every host or application.
- Separate failure domains: avoid placing every collector, Logstash worker, and Elasticsearch node on one host or one storage device.
Troubleshooting by pipeline stage
No events leave the host
- Check that the required input module is loaded and that the source is actually writing to the socket, journal, or file path.
- Validate the ruleset condition. A facility or severity comparison that is too narrow can discard everything before the forwarding action.
- Inspect the action queue and local rsyslog diagnostics for name-resolution, connection, permission, or queue-full errors.
Logstash receives connections but creates no documents
- Verify that the protocol and port match. A TCP sender pointed at a UDP input, or the reverse, will not produce valid events.
- Confirm the codec matches the wire format. A syslog codec cannot correctly parse arbitrary JSON unless the sender emits the expected framing.
- Inspect conditional filters and the malformed-event route. A condition that never matches can bypass the Elasticsearch output.
Elasticsearch rejects output
- Read the Logstash error for authentication, authorization, certificate, index, or mapping failures.
- Confirm the username, password, HTTPS URL, certificate authority path, and certificate name.
- Check that the ingestion account can write the target index and that the index template or lifecycle policy is compatible with the fields being sent.
Events are delayed or missing during an outage
- Inspect rsyslog and Logstash queue depth, queue files, retry counters, and disk usage.
- Determine whether the sender used UDP, which cannot provide reliable retransmission feedback.
- Compare source timestamps with collector and Elasticsearch timestamps to identify clock skew rather than transport loss.
Documents are duplicated
- Look for reconnects, timeout retries, or replay from a persistent queue.
- Use stable event identifiers and an idempotent indexing strategy if duplicates are unacceptable.
- Do not delete duplicates blindly until you know whether they represent legitimate repeated events from the source.
Production checklist
- Inputs are limited to required modules and listeners.
- Rulesets forward only approved facilities, severities, hosts, and programs.
- TCP is selected where loss is unacceptable; TLS is enabled for sensitive traffic.
- Rsyslog queues and Logstash persistent queues have explicit size, disk, and alert limits.
- Logstash parsing failures are tagged and routed for diagnosis rather than dropped.
- Elasticsearch output uses HTTPS, a trusted certificate authority, and least-privilege index permissions.
- Credentials and private keys are protected from configuration repositories and broad filesystem access.
- Index rollover, retention, shard sizing, and replica policy are documented.
- Monitoring covers queue depth, rejected documents, parse failures, certificate expiry, disk usage, and indexing pressure.
- Outage recovery, duplicate handling, clock skew, certificate rotation, and rollover have been exercised in the target environment.
What this architecture gives you
Rsyslog keeps collection close to the source and can continue buffering when downstream services are unavailable. Logstash centralizes the expensive, changeable work of parsing, enrichment, and routing. Elasticsearch provides the indexed store and search layer, with security controls that must be configured rather than assumed. The correct TCP-versus-UDP choice, queue policy, TLS configuration, and index lifecycle depend on your loss tolerance, data sensitivity, event volume, and recovery objectives—not on a universal benchmark.
