Frequently Asked Questions

DarkTrend.com is a tool designed to uncover the hidden (idiosyncratic) price trends of various financial instruments (stocks, indices, ETFs and currencies).

Idiosyncratic price trends refer to patterns and movements in prices that are unique to a specific instrument, rather than being driven by overarching market trends or sentiments.

External market “noise” obscures these trends, making them difficult to detect.

When you observe charts for stocks within the same sector, highly liquid assets, currencies, or cryptocurrencies, you’ll often find they move in sync. They rise and fall together, reflecting the overall trends of their sector, industry, or the broader market.

Statistically speaking, these assets are highly correlated. This shared movement, which we refer to as “noise,” can make it challenging to identify when a specific stock, ETF, or cryptocurrency is beginning to gain momentum independently.

Though it may appear as if we are buying or selling individual stocks, ETFs, currencies, or cryptocurrencies, what we’re actually trading are the overarching trends that influence the entire sector, industry, or market. This is especially true for cryptocurrencies, where their movements are largely influenced by two major players: ETH and BTC.

At DarkTrend.com, we consider this shared, external influence as noise. By removing this noise, we can reveal the true, hidden trajectory of an asset, which we call the “dark trend” - an analogy to dark matter. This dark trend represents the individual momentum of the asset, independent of the broader market trends.

Mathematically speaking, we perceive the fluctuations of any financial instrument at a particular moment as the sum of its individual trend (the “dark trend” ) and an external influence (the “noise”), which is scaled by a coefficient known as Beta. In equation form, it’s expressed as:

𝑆𝑡𝑜𝑐𝑘\𝐸𝑇𝐹\𝐶𝑟𝑦𝑝𝑡𝑜_t = 𝐷𝑎𝑟𝑘 𝑇𝑟𝑒𝑛𝑑 𝑜𝑓 𝑆𝑡𝑜𝑐𝑘\𝐸𝑇𝐹\𝐶𝑟𝑦𝑝𝑡𝑜_t + “𝐸𝑥𝑡𝑒𝑟𝑛𝑎𝑙 𝑁𝑜𝑖𝑠𝑒”_t × β_t

The Beta coefficient isn’t constant; it changes over time and can sometimes shift dramatically. Therefore, we say Beta is dependent on ’t’ (time). Rearranged, it can also be expressed as:

𝐷𝑎𝑟𝑘 𝑇𝑟𝑒𝑛𝑑 𝑜𝑓 𝑆𝑡𝑜𝑐𝑘\𝐸𝑇𝐹\𝐶𝑟𝑦𝑝𝑡𝑜_t = 𝑆𝑡𝑜𝑐𝑘\𝐸𝑇𝐹\𝐶𝑟𝑦𝑝𝑡𝑜_t — “𝐸𝑥𝑡𝑒𝑟𝑛𝑎𝑙 𝑁𝑜𝑖𝑠𝑒”_t × β_t

Individual stocks are part of a larger hierarchy—they’re components of both a broader national market and a specific industry or sector. This means that if the national market or the industry/sector is experiencing an upward or downward trend, the individual stock is likely to follow suit. The degree of this influence is determined by the beta coefficient (BetaBroad, BetaIndustry). In terms of external market “noise”, we use quotes from the SPY ETF for the U.S. market as a source. For industry or sectoral noise, we consider the most liquid and closely related sectoral or industrial ETFs to the individual stock. Through our platform, we offer the ability to observe the dark trend in both these contexts.

Unlike individual stocks, ETFs primarily derive their external “noise” from the broader national market, represented by the SPY. Occasionally, there can be a dual influence—for instance, gold stock ETFs are influenced by both the general market and the specific gold market. However, such situations are generally regarded as exceptions.

At present, the cryptocurrency market is somewhat “flat” as there are no distinct industrial or sectoral clusters. The crypto universe primarily revolves around two central hubs: Bitcoin (BTC) and Ethereum (ETH). However, this landscape is evolving quickly.

Beta coefficients are calculated using a wide variety of statistical and computational tools that are currently available in the fields of science and computational finance.

Our models take into account factors such as heteroskedasticity, time-varying dynamics, and hyperbolic decay. Machine Learning models like random forests and xgboost are used for determining hyperparameters such as the “forgetting rate” and various internal coefficients.

All these calculations use High-Frequency Trading (HFT) data and consider the impacts of bid-ask bounce. As a result, our beta coefficients are high-precision and time-varying HFT betas, computed at the close of each trading day to the greatest extent possible in a fast-changing stochastic environment. The primary goal is to minimize the pseudo-R2 on an out-of-sample basis.

R2, also known as “R-squared” or the coefficient of determination, is a statistical measure that ranges from 0 to 1.0 (or 0% to 100%). It quantifies the extent to which external factors or influences (referred to as “external noise”) explain the behavior of an individual stock, ETF, or cryptocurrency.

If R2 is approximately 1.0 or 100%, it indicates that the financial instrument doesn’t exhibit individual behavior (i.e., there’s no hidden trend, and its idiosyncratic volatility is zero), and all fluctuations in its price are solely due to external circumstances.

Conversely, if R2 equals 0 (which essentially suggests Beta equals 0), then the instrument behaves independently, unaffected by external factors. When the trend isn’t concealed, or “dark”, price quotes are used for analysis.

Darktrend uses pseudo R2, which is a measure of goodness-of-fit that accounts for the fact that the model fits to an adjusted “noisier” price series than the actual. Pseudo R2 shares the same interpretative value as the traditional coefficient of determination, but its calculation method is significantly more robust.

This involves:

1. Using absolute ranges instead of quadratic ones
2. Accounting for bid-ask spreads, ensuring that R2 is not inflated for low-liquidity instruments
3. Performing calculations on an intraday basis using high-frequency trading (HFT) data

All computations are done using a 100-day rolling window, which ensures that the R2 doesn’t remain static and is continuously updated to reflect any changes in external factors or influences.

Absolutely not. We understand that academic and public articles often report in-sample estimates, which can be misleading and lead to overfitting. When determining the values of R2, beta, and the dark trend at any particular point, we strictly use market data that is available up to that point. This is done using a rolling out-of-sample (OOS) methodology which helps avoid common issues such as overfitting, overestimation, and excessively optimistic expectations.

R2 is incorporated into the DarkTrend Screener. We believe that an R2 close to 50% (0.5) is optimal for identifying dark trends. At this level, the stock chart contains enough “external noise” or factors to mask individual trends (dark trends) from public view, while still allowing these hidden trends to exist. In practice, we are interested in instruments with an R2 ranging between 30% and 70% (0.3 to 0.7).

Instruments with a high R2 (>0.8) are of particular interest; these are typically highly correlated ETFs. By closely monitoring significant shifts in their dark trends, we can gain insights into the ‘internal workings’ of asset management companies, such as portfolio restructuring and other uncommon occurrences.

Instruments with an R2 value less than 0.1 aren’t typically useful when it comes to detecting and identifying dark trends. However, these are ideal for creating diversified low-beta portfolios with minimal risk.

The DarkTrend Screener & Explorer is a combined tool designed to help you first filter and then visually explore financial instruments.

• Purpose: It identifies instruments best suited for analyzing hidden (dark) trends, particularly where these trends diverge significantly from visible price action.
• Key Metrics: Filtering is based on metrics like R2, Betas, asset class (stocks, ETFs, leveraged ETFs, etc.), and crucial proprietary liquidity metrics:
  • LiqRS: Compares daily volatility to the average Bid-Ask spread.
  • LiqRS-R2: Compares idiosyncratic (individual) volatility to the average Bid-Ask spread.

LiqRS-R2: This proprietary metric is valuable even beyond the DarkTrend concept.

It pinpoints assets that uniquely combine highly individual price behavior (low market correlation) with strong tradeability.

Instruments scoring high on LiqRS-R2 are often compelling candidates for both short-term trading and long-term investment strategies, as they measure how significantly individual volatility surpasses typical trading costs (bid-ask spread).

• Purpose: Once you’ve filtered a list, the integrated Explorer allows you to rapidly visualize both the standard price chart and the corresponding dark trend for each selected instrument.
• Fast Visual Review: It’s optimized for quick screening using swipe gestures or keyboard arrows (← / →).
• Intelligent Trendlines: A key feature is the trendlines anchored simultaneously to extrema on both the price chart and the dark trend. This design makes spotting key divergences—situations where the hidden trend moves counter to the visible price — much easier and more intuitive.
• Saved Analysis: Registered users can save their drawn trendlines for future reference.

• Visualize the relationship between any two Screener metrics. Use the interactive “brush” tool to select specific clusters of instruments on the plot for deeper investigation into assets with interesting combined properties.

The Screener & Explorer embodies a “filter-then-view” concept. Use the Screener’s powerful metrics to isolate a promising subset of instruments, then efficiently conduct visual analysis with the fast-scrolling Explorer. (Important Note: The selected time-zoom range persists as you swipe through charts, ensuring consistent historical comparisons during rapid review.)

The process of removing noise from financial instruments relies on the assumption of a highly asymmetrical cause-and-effect relationship. Put simply, eliminating the “noise” from GOOG by using the sector ETF XLC is logical only if the impact of XLC on GOOG is significantly greater than the influence of GOOG on XLC.