Imagine a typical evening right before launching an online store. You carefully hook up payments through the Payme Business Merchant API, run a test order for 15,000 UZS, see the status flip to "paid" in the database, and everything looks great. Then you run the automated tests in the Payme sandbox, and they fail on the very first run.
What's the catch? The sandbox enforces a rule that developers often miss when skimming the docs: it sends CreateTransaction, PerformTransaction, and CancelTransaction twice in a row and expects the second response to match the first byte for byte. A straightforward integration breaks on a single harmless line: perform_time: Date.now(). On the second request, the clock has ticked forward by a couple hundred milliseconds. The response changes, and the test turns red.
The sandbox isn't just being difficult. It simulates real production: your server processed the payment, but the response got lost on the way back due to a network drop. Payme Business retries the exact same request and expects the exact same reply. If your backend isn't ready for that, the provider flags the payment for manual review, and your support team gets the worst ticket imaginable: "My money was charged, but my order isn't paid."
Let's see how to build a fault-tolerant payment system on PostgreSQL and TypeScript for two different providers at once—Payme and Click SHOP API—and back it up with nightly reconciliation. The examples use Uzbek sums and tiyins, but the database design, race-condition handling, and mismatch checks apply directly to any payment gateway.
Five Rules of the Payme Contract and the SELECT-Before-INSERT Trap
Before opening your code editor, let's lock down five strict rules from the Payme documentation:
- The protocol is JSON-RPC 2.0 over
HTTP POST. You must always reply with HTTP status200, even if the order isn't found or the amount doesn't match. Business errors go inside theerrorfield in JSON (with a numeric code and a localizedmessageobject forru,uz, anden). Any other HTTP code like400or404looks like a system failure-32400to the provider, not a business rejection. - The amount arrives as an integer in tiyins (
15,000UZS becomes1,500,000). Never usefloator floating-point math. - Timestamps are 13-digit Unix timestamps in milliseconds.
- If the response to
CreateTransactionorPerformTransactionis lost, Payme retries the call with the same parameters. - A transaction in the "created" state lives for exactly 12 hours (
43,200,000ms), after which it must be canceled.
Point four leads to a key insight: Payme's external transaction ID (params.id) is already an idempotency key. Idempotency means an operation produces the exact same result on repeated calls with the same input without corrupting system state. You don't need to invent your own key on top of params.id.
Your first instinct might be to run a SELECT in your code, check if the record exists, and run INSERT if it doesn't. On your laptop, that works fine. In production, two identical webhooks arrive milliseconds apart on two separate Node.js workers. Both run SELECT, both see nothing, and both proceed to INSERT. The bug wins the race. The database itself must enforce duplicate protection.
A Unified Foundation in PostgreSQL: The provider_txn Table
It's much better to store gateway transactions in a dedicated table rather than adding columns to orders. A single order can easily have multiple payment attempts: a customer starts paying via Payme, changes their mind, comes back an hour later to pay with Click, and then asks for a refund. When you investigate an incident, you'll want the full history of every attempt.
create table provider_txn (
id serial primary key,
-- 'payme' | 'click'
provider text not null,
-- params.id from Payme or click_trans_id from Click
external_id text not null,
order_id bigint not null references orders(id),
amount_tiyin bigint not null,
-- 1 created, 2 performed, -1 canceled before, -2 canceled after
state smallint not null,
reason smallint,
-- params.time from Payme, needed for GetStatement reconciliation
provider_time bigint,
-- The three timestamp fields below are written strictly once
create_time bigint not null,
perform_time bigint not null default 0,
cancel_time bigint not null default 0,
raw_last jsonb,
unique (provider, external_id)
);
-- An order cannot have a second pending Payme transaction
create unique index one_pending_per_order
on provider_txn (order_id)
where provider = 'payme' and state = 1;
Two core safeguards live right inside PostgreSQL and work just as reliably whether you run one app container or twenty replicas. The composite unique constraint (provider, external_id) blocks concurrent duplicates. Meanwhile, the partial unique index one_pending_per_order (an index with a where condition that only checks uniqueness for matching rows) stops an order from getting a second active Payme transaction in the pending state.
Take a close look at the three timestamp columns: create_time, perform_time, and cancel_time. That "written strictly once" comment isn't just decoration. Once a transaction enters a given state, its timestamp is saved in the row, and all future retries return that stored value instead of calling Date.now() again.
Payme State Machine, Transactional Outbox, and Auth
In the Payme protocol, a transaction has exactly four states and four allowed transitions:
PerformTransaction
[none] --Create--> (1) -----------------> (2) performed
| |
Cancel/timeout | | Cancel (not fulfilled)
v v
(-1) (-2)
Here, 1 means the transaction is created and waiting for confirmation, 2 means the payment is completed, -1 means canceled before completion, and -2 means canceled after completion. Any attempt to jump outside this diagram must return JSON-RPC error -31008 ("unable to perform operation"). If a transaction expires after the 12-hour timeout, move it to -1 with reason = 4.
Let's see how the CreateTransaction handler works. Inside a database transaction, we first look up the row using SELECT ... FOR UPDATE, which places an exclusive lock on the row until commit so concurrent requests wait instead of modifying it at the same time.
const TIMEOUT_MS = 43_200_000;
async function createTransaction(
p: CreateParams,
): Promise<RpcResult> {
return db.tx(async (tx) => {
const existing = await tx.oneOrNone<ProviderTxn>(
`select * from provider_txn
where provider = 'payme' and external_id = $1
for update`,
[p.id],
);
if (existing) {
if (existing.state !== 1) throw rpcError(-31008);
if (Date.now() - existing.create_time > TIMEOUT_MS) {
await cancel(tx, existing, -1, 4);
throw rpcError(-31008);
}
// Retry: return the exact same stored timestamp
return {
create_time: existing.create_time,
transaction: String(existing.id),
state: 1,
};
}
const order = await tx.findOrderForUpdate(p.account.order_id);
if (!order) throw rpcError(-31050, 'order_id');
if (order.amount_tiyin !== p.amount) throw rpcError(-31001);
if (order.status !== 'awaiting_payment') {
throw rpcError(-31008);
}
const row = await tx.insertOrNull('provider_txn', {
provider: 'payme',
external_id: p.id,
order_id: order.id,
amount_tiyin: p.amount,
state: 1,
provider_time: p.time,
create_time: Date.now(),
});
if (!row) {
// Unique conflict: check if this is a duplicate params.id
const twin = await tx.findTxn('payme', p.id);
if (twin) {
return {
create_time: twin.create_time,
transaction: String(twin.id),
state: twin.state,
};
}
// Otherwise one_pending_per_order fired: order is already busy
throw rpcError(-31008);
}
return {
create_time: row.create_time,
transaction: String(row.id),
state: 1,
};
});
}
This shows why one_pending_per_order is so handy. The Payme sandbox tests a tricky edge case: an order is already waiting for payment on one transaction, and a second request arrives for the same order. The docs phrase this loosely (you should reserve the order until payment or timeout), while the sandbox specifically expects error -31008. The partial index turns that whole rule into a standard insert conflict: if no row matches params.id, the busy-order index fired, and you cleanly throw -31008.
Now comes the most dangerous part of the entire integration: PerformTransaction. This is where production systems lose money.
async function performTransaction(
p: { id: string },
): Promise<RpcResult> {
return db.tx(async (tx) => {
const t = await tx.lockTxn('payme', p.id);
if (!t) throw rpcError(-31003);
if (t.state === 2) {
// Retry: return stored perform_time, never Date.now()
return {
transaction: String(t.id),
perform_time: t.perform_time,
state: 2,
};
}
if (t.state !== 1) throw rpcError(-31008);
if (Date.now() - t.create_time > TIMEOUT_MS) {
await cancel(tx, t, -1, 4);
throw rpcError(-31008);
}
const performTime = Date.now();
await tx.update('provider_txn', t.id, {
state: 2,
perform_time: performTime,
});
await tx.markOrderPaid(t.order_id, t.id);
await tx.insert('outbox', {
type: 'order.paid',
order_id: t.order_id,
});
return {
transaction: String(t.id),
perform_time: performTime,
state: 2,
};
});
}
Look at the three lines before return. Updating provider_txn, marking the order paid, and writing an event to the outbox table all happen in a single PostgreSQL transaction. What if you split those steps? Suppose the handler writes state = 2, then makes a network call to send an SMS receipt, hits a gateway timeout, and crashes before updating the order. Payme retries PerformTransaction, your code sees state === 2, happily returns success, and the order stays unpaid forever.
To prevent that, use the Transactional Outbox pattern. Instead of making direct network calls inside a payment webhook, you write a task row to the outbox table in the same database transaction. A separate background worker reads new rows from outbox later and safely sends SMS messages, fiscal receipts, and warehouse notifications.
Architecture diagram of the Transactional Outbox pattern: writing state and events in a single database transaction (Source: microservices.io)
The CancelTransaction logic is also straightforward, with one subtle catch.
async function cancelTransaction(
p: { id: string; reason: number },
) {
return db.tx(async (tx) => {
const t = await tx.lockTxn('payme', p.id);
if (!t) throw rpcError(-31003);
if (t.state === 1) {
await cancel(tx, t, -1, p.reason);
} else if (t.state === 2) {
// -31007 only if goods/services were actually delivered
if (await tx.isOrderFulfilled(t.order_id)) {
throw rpcError(-31007);
}
await cancel(tx, t, -2, p.reason);
}
// For already canceled (-1 and -2), return stored timestamp
const fresh = await tx.getTxn(t.id);
return {
transaction: String(fresh.id),
cancel_time: fresh.cancel_time,
state: fresh.state,
};
});
}
Code -31007 means "the product or service has already been provided and cannot be canceled." A common mistake is checking this against an "order paid" flag. Paid and fulfilled are two different things. If the customer already received a subscription or the courier handed over the box, return -31007. If the order is just sitting in paid status and hasn't left the warehouse yet, you can move the transaction to -2.
That leaves two helper methods and authorization. CheckTransaction simply returns fields from the provider_txn row (or -31003 if missing). Meanwhile, teams often forget about GetStatement at launch, even though Payme uses it to reconcile with your store. It returns an ascending list of transactions whose creation time on Payme's side (params.time from CreateTransaction) falls within from <= time <= to, excluding records where CreateTransaction failed. That's why we store provider_time separately from our local create_time—never mix them up.
Payme sends authorization in the Authorization: Basic ... header, where the password is your merchant cashbox key. Return -32504 on an invalid key; the sandbox tests this too. Don't compare the secret with ===: standard string comparison stops at the first mismatched character, which theoretically leaks timing data (a timing attack). Node.js provides crypto.timingSafeEqual to compare buffers in constant time.
Why the Payme Template Breaks on Click SHOP API
Once Payme works, it's tempting to copy the handlers, rename a few fields, and hook up a second gateway—Click—in one evening. If you try that directly, the Click integration will break. The two protocols differ in almost every detail.
Engineering diagram comparing Payme and Click protocols with JSON-RPC and URL-encoded formats, amount calculation, and response codes
| Parameter | Payme Merchant API | Click SHOP API |
|---|---|---|
| Request format | JSON-RPC 2.0 (HTTP POST, always 200) | POST, application/x-www-form-urlencoded |
| Auth / signature | Basic Auth (cashbox key) | sign_string (MD5 with SECRET_KEY) |
| Amount | Integer in tiyins (1500000) | Decimal string in sums ("15000.00") |
| Idempotency key | params.id | click_trans_id |
| Protocol steps | 6 methods (Check, Create, Perform, Cancel, CheckTransaction, GetStatement) | 2 steps: Prepare (action=0) -> Complete (action=1) |
| Retry of completed payment | Same success response | Error -4 ("Already paid") |
The last row is the sharpest trap. In Payme, a repeated PerformTransaction must return the exact same success JSON with the saved perform_time. In Click, a repeated Complete must return error -4. Before you even reach the payment steps, though, it's easy to trip over two other things: signatures and money.
Click's two steps differ by one field in their MD5 signature formula (merchant_prepare_id only appears in step two):
Prepare:
md5(click_trans_id + service_id + SECRET_KEY + merchant_trans_id
+ amount + action + sign_time)
Complete:
md5(click_trans_id + service_id + SECRET_KEY + merchant_trans_id
+ merchant_prepare_id + amount + action + sign_time)
A classic bug hides right here. You must concatenate these fields into a string for hashing in the exact raw format they arrived in the form body, before any number or date parsing. If Click sends amount as "15000.00" and your framework converts it to the number 15000 and back to "15000", the signature won't match, and you'll start rejecting valid payments with -1.
To compare the amount against your database order, convert the string to integer tiyins using BigInt—no Number and no parseFloat—so you avoid surprises like 19.99 * 100 === 1998.9999999999998:
function sumToTiyin(s: string): bigint {
// Accept whole sums and up to two decimal places
const m = /^(\d+)(?:\.(\d{1,2}))?$/.exec(s);
if (!m) throw new ClickError(-2);
const sums = BigInt(m[1]) * 100n;
const tiyins = BigInt((m[2] ?? '').padEnd(2, '0'));
return sums + tiyins;
}
This function splits the string with a regex into whole and fractional parts, pads the fraction to two digits with zeros, and adds them as BigInt. The full set of response codes in the Click SHOP API error docs is compact: 0 (success), -1 (invalid signature), -2 (invalid amount), -3 (action not found), -4 (already paid), -5 (order not found), -6 (transaction not found), -7 (update failed), -8 (error in request from Click), and -9 (transaction canceled).
Prepare, Complete, and Refunds via reversal in Click
In the first step (Prepare, action === '0'), Click asks your backend: is the order valid, does the amount match, and can you fulfill it? This is where your store should reserve inventory.
async function prepare(r: ClickPrepare): Promise<ClickResponse> {
if (!verifySign(r, { withPrepareId: false })) return err(-1);
if (r.action !== '0') return err(-3);
return db.tx(async (tx) => {
const order = await tx.findOrderForUpdate(r.merchant_trans_id);
if (!order) return err(-5);
if (order.status === 'paid') return err(-4);
if (order.amount_tiyin !== sumToTiyin(r.amount)) {
return err(-2);
}
// insert ... on conflict do nothing, then select:
// repeated Prepare returns the same merchant_prepare_id
const t = await tx.insertOrGet(
'provider_txn',
{ provider: 'click', external_id: r.click_trans_id },
{
order_id: order.id,
amount_tiyin: sumToTiyin(r.amount),
state: 1,
create_time: Date.now(),
},
);
if (t.state < 0) return err(-9);
return ok({
click_trans_id: r.click_trans_id,
merchant_trans_id: r.merchant_trans_id,
merchant_prepare_id: t.id,
});
});
}
It uses the exact same provider_txn table as Payme. The composite unique index (provider, external_id) protects against repeated Prepare calls, and the primary key t.id goes back to Click as merchant_prepare_id.
In the second step (Complete, action === '1'), Click sends back your merchant_prepare_id along with the charge result.
async function complete(r: ClickComplete): Promise<ClickResponse> {
if (!verifySign(r, { withPrepareId: true })) return err(-1);
if (r.action !== '1') return err(-3);
return db.tx(async (tx) => {
const t = await tx.lockTxnById(Number(r.merchant_prepare_id));
if (!t || t.external_id !== r.click_trans_id) return err(-6);
// Repeated Complete on a paid txn -> -4 ("Already paid")
if (t.state === 2) return err(-4);
if (t.state < 0) return err(-9);
if (Number(r.error) < 0) {
// Payment failed on Click's side: cancel locally
await cancel(tx, t, -1, null);
return err(-9);
}
await tx.update('provider_txn', t.id, {
state: 2,
perform_time: Date.now(),
});
await tx.markOrderPaid(t.order_id, t.id);
await tx.insert('outbox', {
type: 'order.paid',
order_id: t.order_id,
});
return ok({
click_trans_id: r.click_trans_id,
merchant_trans_id: r.merchant_trans_id,
merchant_confirm_id: t.id,
});
});
}
Pay attention to the Number(r.error) < 0 branch. If the customer had insufficient funds or their bank declined the charge, Click still sends Complete, but with a negative error field. When that happens, you must mark the local transaction as canceled (state = -1) and reply with -9.
Now for a rule from the Click SHOP API notes that teams usually discover only after their first messy incident. Suppose Prepare succeeded, the customer's card was charged, Complete arrives with error = 0—and the last item in stock was sold to someone else a minute earlier. You cannot reply to Complete with an error: the customer's money will be stuck, and the payment will go to manual review. Under Click's contract, after a charge you can only return an error on Complete in two cases: already confirmed (-4) or canceled (-9).
The right move is to reply to Click with success (0) and immediately trigger a reversal (a forced cancellation of a completed payment) via the Click Merchant API: DELETE /v2/merchant/payment/reversal/:service_id/:payment_id. Don't delay this. Reversals have strict limits: the payment must be completed and paid by online card, it can only be reversed in the current billing month (for the previous month, only on the first day of the new month), and the UZCARD system can still reject it. If you catch a mismatch at the end of the month during accounting instead of the same day, an automatic reversal won't work.
Try the interactive simulator below: pick a provider and a failure scenario to compare how Payme and Click react to the same event.
Payment Webhook Simulator: Payme vs Click
When you view both contracts side by side, you immediately lose any desire to cram Payme and Click into one "universal" controller. Only the provider_txn + outbox database schema stays shared. Each gateway's protocol adapter should live separately and strictly follow its own response codes.
Third Layer of Defense: Nightly Reconciliation
Idempotency and PostgreSQL indexes handle race conditions and retries well, but they can't prevent every real-world failure. A process might crash right between committing and sending the HTTP response. A manager might manually flip an order status in the admin panel. A provider might cancel a transaction on their side without sending a webhook. Only regular reconciliation catches these mismatches—and it should run every night, not once a month before closing the books.
First, the job runs four SQL queries checking internal invariants in your database. In a healthy system, all four must return zero rows. Any row returned is an incident ready to investigate:
Isometric architectural diagram of nightly transaction reconciliation comparing a PostgreSQL database with external provider registries
-- 1. Money charged, but order not paid (paid_not_fulfilled)
select t.*
from provider_txn t
join orders o on o.id = t.order_id
where t.state = 2
and o.status <> 'paid';
-- 2. Order paid, but no completed transaction (order_without_txn)
select o.*
from orders o
where o.status = 'paid'
and not exists (
select 1 from provider_txn t
where t.order_id = o.id and t.state = 2
);
-- 3. Double payment on a single order (double_payment)
select order_id, count(*)
from provider_txn
where state = 2
group by order_id
having count(*) > 1;
-- 4. Transactions stuck in state = 1 longer than 12 hours (stuck_pending)
select *
from provider_txn
where state = 1
and create_time <
(extract(epoch from now()) * 1000)::bigint - 43200000;
You might wonder: how could the third case (double_payment) ever happen if we already have one_pending_per_order? Remember its condition: where provider = 'payme' and state = 1. It only prevents duplicates within Payme. If a buyer abandons a Payme payment and pays for the same order five minutes later via Click, the system shouldn't block them. Blocking a customer over a rare edge case is far worse than catching a double payment once every six months in nightly reconciliation and issuing a refund.
After checking internal invariants, reconcile with the external providers. Click's Merchant API has a targeted endpoint to check status by your order ID:
GET /v2/merchant/payment/status_by_mti/:service_id/:merchant_trans_id/YYYY-MM-DD
It expects an Auth: merchant_user_id:digest:timestamp header, where digest = sha1(timestamp + secret_key). Watch out for another unit trap here: unlike Payme's 13-digit milliseconds, Click's Auth header expects a Unix timestamp in seconds. You can call this endpoint specifically for any suspicious orders flagged by queries 1, 2, and 4.
With Payme, the direction is reversed: Payme calls GetStatement to check against you, so your response must match provider_txn. On your side, you compare your database against the daily transaction registry and bank settlements. Here is what the skeleton of the nightly job looks like:
async function nightlyReconciliation(day: Date) {
const findings: Finding[] = [
...(await db.many(Q_PAID_TXN_UNPAID_ORDER))
.map((r) => f('paid_not_fulfilled', r)),
...(await db.many(Q_PAID_ORDER_NO_TXN))
.map((r) => f('order_without_txn', r)),
...(await db.many(Q_DOUBLE_PAYMENT))
.map((r) => f('double_payment', r)),
...(await db.many(Q_STUCK_PENDING))
.map((r) => f('stuck_pending', r)),
];
// For Click, fetch live order status from Merchant API
for (const x of findings.filter((i) => i.provider === 'click')) {
x.external = await click.statusByMti(x.merchant_trans_id, day);
}
const registry = await loadProviderRegistry(day);
// Archive the raw registry before any processing
await archiveRaw(day, registry);
// Match strictly by provider ID, never by amount and time
findings.push(...diffById(await db.txnsForDay(day), registry));
// Open a ticket with an owner for each discrepancy
for (const x of findings) {
await tickets.open(x);
}
}
Two small details in that code save hours of debugging in practice. First, always match operations against the registry using the provider's external ID (params.id or click_trans_id), never by an "amount + timestamp" pair: two purchases for 15,000 UZS happen in the same minute every day in any busy store. Second, archive the provider's registry file in its raw form before running any parsers or filters. When accounting or support asks about a mismatch a week later, you'll pull out the original file from the provider rather than a processed copy.
Production Launch Checklist
Before turning on real money, walk through the checklist below and verify every item in your staging environment.
Checklist Before Launching Payme and Click in Production
- PostgreSQL has the
provider_txntable withunique (provider, external_id)and the partial indexone_pending_per_order—race-condition protection lives in the DB, not inSELECTbeforeINSERT. -
create_time,perform_time, andcancel_timeare written once and returned unchanged on retries;provider_timeis stored separately for Payme'sGetStatement. - The Payme handler always returns
HTTP 200(errors in JSON), checks the 12-hour43,200,000ms timeout inCreateTransactionandPerformTransaction, and verifies the Basic Auth key usingcrypto.timingSafeEqual. - Updating
provider_txn.state = 2, marking the orderpaid, and writing theoutboxtask run in one DB transaction; a separate worker sends SMS, receipts, and webhooks. - Error
-31007in Payme (CancelTransaction) checks actual fulfillment or delivery, not just the "paid" status. - Click verifies its MD5 signature against raw form strings before type conversion, and converts amounts to integer tiyins via
BigIntwithoutfloat. - Repeated
Completecalls in Click return-4, incomingerror < 0cancels the transaction with-9, and out-of-stock after charge returns0followed immediately byDELETE /v2/merchant/payment/reversal/.... - The
test.paycom.uzsandbox passes completely, and thenightlyReconciliationjob with four SQL invariants, raw registry archiving, and ticketing runs on a daily schedule.
Once this setup passes green in the Payme sandbox, survives repeated Complete tests in Click, and produces an empty nightly reconciliation report, your payment backend is ready for real traffic—even when a customer's mobile connection drops for half a second right as their card is charged.

![Node.JS [ru]](/api/digests/it_development/daily/20261122/assets/sources/we-use-js.jpg?v=8a922c4c9193ead8575dd869ae7a5dafe5bd8590efaaa20c7d3842b3cedee257)
