TL;DR

  • Rules-based surveillance throws off false positives and can be worked around once traders find the thresholds.
  • Detection duties now sit with brokers, not just venues: MAR Article 16 covers any firm arranging or executing transactions.
  • Unsupervised models learn normal order book behavior and flag departures from it.
  • Anomaly scores prioritize the analyst’s queue; the rules layer and the compliance officer still own the audit trail.

A lot has been made of AI’s relatively new capabilities in synthesizing information and presenting it in novel, human-like ways. But its distinctively non-human pattern-recognition abilities are also revolutionizing how we monitor domains that are far too complex for us, providing early warning signals that would otherwise have been lost in the noise.

Market surveillance used to be the domain of exchanges and tier-one banks, the only institutions with the resources to sift through order flow at any depth, whether policing their own markets or looking for an edge in them. Those are two very different jobs, and until recently, both of them demanded the same expensive infrastructure. The barriers to entry have been lowered by this new generation of AI trade surveillance tools, and today, AI market surveillance is making pattern detection accessible to mid-size and retail brokers.

The issue with rules-based surveillance

In the past, rules-based metrics were the only tools available for spotting deviations along static dimensions, like flagging orders that are canceled within a certain period, or orders placed on both sides of an order book within a given time. The idea is that patterns of market manipulation can be isolated by combining these flags.

The problem with static approaches like these is that they generate high rates of false positives. Different market participants exhibit different trading patterns, and it’s hard for one-dimensional models to differentiate between manipulation and legitimate strategies, such as market makers and HFT algos placing and canceling huge volumes of orders in their ordinary run of business. There’s also the issue that static flags, once set, can be triangulated by diligent traders and then simply worked around.

Processes looking outward at market dynamics, rather than individual trading practices, also have some of the same shortcomings. They can be too simplistic in their “if this, then that” readings of market behavior and just as vulnerable to being blindsided by sudden shifts in regime that go below the radar.

CFTC v. Navinder Singh Sarao

CFTC v. Navinder Singh Sarao is an interesting case in point when it comes to market surveillance. Sarao was charged in 2015 with market manipulation and incurred penalties in the tens of millions of dollars for running spoofing algorithms on E-mini S&P 500 futures contracts between 2009 and 2014.

The 2010 flash crash has subsequently been partly attributed to this activity, which went unchallenged by exchange surveillance practices for years as it was more sophisticated than the spoofing detection measures taken to identify it. CME had, in fact, contacted Sarao about his order activity as early as 2009 and warned him about the new anti-spoofing provision in 2010, but assembling evidence of intent took another five years.

Why surveillance became a broker’s problem

It’s tempting to read the Sarao affair as a story about exchanges, but the regulatory response ran in the opposite direction, pushing detection obligations outward and downward to the firms that handle order flow.

In the US, Section 747 of the Dodd-Frank Act had written spoofing into the Commodity Exchange Act in July 2010, which is precisely the provision CME cited when it wrote to Sarao that October. In Europe, Article 16 of the Market Abuse Regulation goes further still, requiring any person professionally arranging or executing transactions to establish and maintain systems and procedures capable of detecting and reporting suspicious orders and transactions, with cancellations and modifications included. Where a reasonable suspicion is formed, the firm files a Suspicious Transaction and Order Report with its national regulator. The UK onshored the same regime after Brexit and layers its own FCA Handbook requirements on top of it.

That phrase, “person professionally arranging or executing transactions,” isn’t confined to trading venues or tier-one banks. A retail broker receiving and transmitting client orders sits squarely inside the definition, which means the job of spotting manipulation in its own order flow belongs to the broker rather than to somebody further up the chain. Add the monitoring that best execution obligations imply under MiFID II, along with the growth in retail trading volumes over the past few years, and a mid-size brokerage now carries a market integrity burden that would have been somebody else’s problem not so long ago.

Unsupervised machine-learning surveillance

The big difference with the AI technologies now being brought to bear on this issue is that instead of looking for individual triggers or sequences thereof, unsupervised machine learning models can observe and acquire a baseline of, for example, how order book dynamics appear under “normal” conditions, and flag statistical deviations from these norms. In this way, deviations can be spotted even when they don’t correspond to any single predefined rule.

By also being sensitive to sequences of events rather than just individual points, these deep learning models are able to view markets at a much higher resolution. This means that activity that was previously lost in the avalanche of trading can now be brought into high relief. This works both when trying to identify whether market participants are playing by the rules and when analyzing price action for telltale clues about what’s likely to happen next.

Furthermore, wash-trading detection, layering detection, and manipulation through the use of multiple accounts also come into view with these new tools due to their ability to analyze networks, rather than just individual accounts and positions against lists of rules. Being able to track account ownership, counterparties that trade with each other to the exclusion of other participants, and unusual clustering can help bring relationships to light that would otherwise have been invisible at the individual account level.

The compliance desk’s side of the problem

None of this counts for much if it simply produces more alerts, and this is where the false positive rate becomes an operating cost. Every alert a system raises has to be triaged by somebody, worked through an investigation, documented, and then either escalated or closed out with reasons. A surveillance system that doubles its coverage while tripling its noise has made the compliance team’s job harder rather than easier, and smaller teams feel this most acutely because they have the least slack to absorb it.

There’s a further wrinkle, which is that the record matters as much as the alert. Under the technical standards accompanying MAR, firms have to retain the analysis behind suspicious orders and transactions, including the ones they examined and decided not to report, and keep that audit trail available to the regulator on request. Those same standards anticipate a degree of automation while insisting on an appropriate level of human analysis.

This is why the more useful framing puts AI in front of rules-based surveillance. An anomaly score is a prioritization signal rather than a verdict, telling a compliance officer where to look first among ten thousand order sequences that all technically satisfy the rules. The rules layer still produces the auditable logic a regulator expects to see, and the compliance officer still signs the report. What changes is that the analyst’s attention gets pointed at the order flow most likely to repay it.

Introducing Grenadier from dxFeed

As with everything in the AI space right now, it all comes down to the quality of the data at your disposal, which is why dxFeed’s Historical Data Lake (HDL), a huge repository of high-resolution market data, is the ideal foundation for a host of market intelligence and surveillance tools the company is bringing to market.

dxFeed Grenadier is one of these projects. Designed to make sophisticated market intelligence available to a far broader subset of financial services firms and other market participants, it provides real-time surveillance of market anomalies, such as unusual order placement, to detect significant price moves, trend changes, and possible halts in trading.

dxFeed’s HDL and dxLink technologies ensure seamless integration and efficient training on huge historical datasets, as well as effective performance in real time. It provides trade surveillance for brokers and exchanges, as well as furnishing traders with a view of market anomalies, offering Level 2 order book analysis, comprehensive order book reconstruction, and allowing for models to be fine-tuned to specific outliers that are important to the firms or individuals in question.

The technology is flexible and serves a variety of use cases, including applying advanced risk management techniques, detecting market shifts in advance of other participants, optimizing trade execution, generating novel market insights for clients and partners, and developing custom models.

These features are just as important for institutions, which can use the granular signals provided to inform risk management practices or existing workflows geared toward fraud detection. Professional traders get access to a far more surgical analysis tool than they’re used to, allowing them to gain a competitive advantage by furnishing their existing strategies with order book anomaly detection AI, as well as the prospect of customizing these complex models to their individual needs.

Crucially for brokers, all of this is now considerably easier to reach. Grenadier is available for integration via DXtrade, Devexperts’ multi-asset trading platform, which puts anomaly scores in front of trading desks as well as risk and compliance teams without anyone needing to open a separate console or stand up a parallel surveillance stack. It sits naturally alongside the platform’s existing risk toolkit and alongside Devexperts’ AI client engagement agent, which already handles the trader-facing side of the conversation and can carry the message when conditions shift.

These developments are enormously exciting because they mean the cutting edge that’s available to small firms, or even individual investors, is becoming more accessible and more affordable, really democratizing high-quality real-time market information.