What does this solve?
There are always gaps between services. The application fell into the form on the website - the manager will see it when he opens the mail. The payment went through - there is no record in the CRM, because no one wrote a link. The report is collected manually from three uploads, and this is done by one person who sometimes goes on vacation. Usually the cracks are plugged with scripts according to a schedule: the script is on one machine, the author remembers about it, there is no launch log, and the failure is detected by the lack of results after a week. n8n collects such bundles in one place: a canvas with a chain of steps, a common storage of accesses, a history of each launch with data at the input and output of each step, and a repeat button from the point of failure. It is installed on your own server, so tokens and client data do not go to an external vendor.
How it works
Application on Node.js. Launched by a container or via npx, listens to port (default 5678), serves a web editor and accepts incoming HTTP requests. The working unit is workflow, a directed graph of nodes.
Trigger
The first node specifies when the chain starts: a schedule (Schedule Trigger), an incoming HTTP request (Webhook), the appearance of a new line, letter or file in an external service (the connector polls it itself), a message from the queue, a call from another workflow. Until the workflow is switched to the Active state, only the test mode from the editor works.
Data
An array of elements travels along the connections between the nodes. Each element has two parts: json —field structure, binary — files. A node receives N elements and returns M, so most often there is no need to draw a cycle in the graph: the nodes process the entire array at once. Field references are written using expressions in double curly braces: {{ $json.email }} —field of the current element, {{ $('Webhook').item.json.id }} —field from the result of a specific node by its name. Renaming a node breaks such links - this is the most common reason for a suddenly empty field.
Steps
In addition to ready-made connectors for popular services, there is a universal set: HTTP Request (any REST API for which there is no connector), IF and Switch (branching), Merge (connecting branches), Loop Over Items (batch processing), Wait (pause or wait for an external signal), Code (JavaScript, there is also Python via Pyodide), Execute Workflow (calling a subprocess), Respond to Webhook (synchronous response to the calling party).
Access
Logins, tokens and OAuth connections are located separately from workflow, in the Credentials section, and are encrypted with a key. The key is created upon first launch and stored in a configuration file in the data directory; it can also be set to a variable N8N_ENCRYPTION_KEY. Transferring an instance to a new server without transferring the key leads to one result: all saved accesses are unreadable, you have to start it again manually.
Storage and scale
By default, the database is an SQLite file in the data directory. For a production installation, we use Postgres: SQLite encounters locks during parallel runs and does not survive history growth well. The normal mode of operation is one process for everything. Queue mode (EXECUTIONS_MODE=queue) separates the roles: the main process with the editor, Redis as a broker, separate worker processes for execution and a separate process for receiving webhooks. This also cures the situation when one long launch blocks the rest.
Errors
Each node has retry settings: Retry On Fail, number of attempts, pause between them. The error behavior can be configured separately - stop the launch, continue further, or move the element to a separate error branch. The entire workflow is assigned an Error Workflow: a separate chain into which an object with a description of the crash is transferred (workflow name, node, error text, launch link). A notification is placed in it in messenger or mail, otherwise no one will know about the falls.
Two webhook addresses
The Webhook node has two URLs. Test (/webhook-test/...) responds only while the button for waiting for a request is pressed in the editor - it is for debugging. Combat (/webhook/...) works only for activated workflow. Behind the reverse proxy you need to set the external address of the variable WEBHOOK_URL: otherwise the interface will show the internal address of the container, and the event provider will have a broken link.
Useful scripts
- Accepting applications: form webhook → normalization of fields → creating a deal in CRM → message in the work chat with a link to the deal.
- Night reconciliation: schedule → unloading from the database and from the payment gateway → comparison of amounts → letter only if there is a discrepancy.
- Parsing incoming documents: letter with attachment → text extraction → query to the language model for the structure → row to table and file to storage.
- Monitoring external dependencies: polling statuses on a schedule → recording the result → alert when several checks in a row are unsuccessful.
- Internal service binding: one webhook input that distributes events into subprocesses via Execute Workflow and responds to the caller synchronously via Respond to Webhook.
Limitations
The license is not open in the usual sense. The sources are available, internal use and work for the client are allowed, but resale of n8n as your own cloud service is not. Some of the enterprise-level features (single sign-on, differentiation of rights, environments, external secrets) live in paid editions.
Memory. Elements travel through nodes in RAM. Uploading hundreds of thousands of lines or several large files in one run hits the container limit, and the process is killed by the OOM killer. It can be cured by batch processing via Loop Over Items and the mode of storing binary data on disk, but the ceiling remains.
Not a replacement for ETL and not a replacement for code. Complex logic with dozens of branches on the canvas is worse to read than a hundred lines in any language. A graph of fifty nodes is debugged slowly, and versioning is slow git gives an unreadable JSON differential: code review of such an edit is almost impossible.
Service. We need HTTPS, database backups along with the encryption key and trimming the execution history - otherwise the executions table grows until the disk runs out (variables are responsible for this EXECUTIONS_DATA_PRUNE and age limit for entries). Version updates periodically change the behavior of nodes, so the image version is fixed, and the update takes place first on a test instance.
External community nodes are installed from npm and run with full process rights. Someone else's package gets access to the same credits as everything else.
Poll triggers are not instantaneous: the connector goes to the service according to a schedule, and the delay is equal to the polling interval. An instant response only happens where the service can send a webhook itself.
How to check the result
The first diagnostic screen is a list of Executions. For each launch you can see the status, duration, culprit node and data at each step. A failed launch is repeated by the button, and you can repeat it with the saved input data, rather than pulling the source again.
During the assembly of the chain, Pin Data helps: the result of the trigger is fixed, and then the steps are raced on the same data without resorting to an external service. A separate node is launched using Execute Node, without running the entire chain.
The combat webhook is checked externally, not from the editor: curl -X POST at the production address with the real body of the request, then checking that a new launch has appeared in the history. If there is no launch, check whether workflow is activated and whether the address matches what is registered with the event provider.
The liveliness of the service itself is checked by the endpoint /healthz, process status - container logs. In queue mode, it is separately checked that workers are actually processing tasks: runs should not accumulate in the waiting status.
Error Workflow is tested by intentional failure - temporarily inserting an incorrect token and making sure that the notification has arrived and it reads the chain name and error text. A silent error handler is worse than no error handler.
Idempotency is checked by repetition: the same request is sent twice and there should be one record left on the target system, not two. If a duplicate appears, a check is added to the chain for an existing object using a foreign key before creation.
Once a month it makes sense to look at the size of the database and the number of stored runs: the growth of history is the quietest reason why an instance one day stops starting.