Pandas (CSV or other data)¶

Q: How do I backtest with my own CSV data?

Load the file into a pandas DataFrame, wrap it in a Data object with its Asset and timestep, and put it in a pandas_data dictionary keyed by the asset. Then pass PandasDataBacktesting and pandas_data=pandas_data to run_backtest(), as in the full example above.

Q: What columns does my DataFrame need?

It needs open, high, low, close and volume. Upper case names such as Close are changed to lower case for you. The index should be the date and time. If it is not, LumiBot looks for a column named datetime, date, timestamp or time and uses that as the index.

Q: Which timesteps does the Pandas backtester support?

Use timestep="minute" for minute bars or timestep="day" for daily bars. The Data class also accepts "hour". The timestep must match the bars in your file.

Q: Can I select the Pandas backtester with BACKTESTING_DATA_SOURCE?

No. Pass PandasDataBacktesting in your code instead. If BACKTESTING_DATA_SOURCE is set to another source, it wins over the class in your code, so leave it unset (or set it to none) when you use your own data.

NOTE: Please ensure you have installed the latest lumibot version using ``pip install lumibot –upgrade`` before proceeding as there have been some major changes to the backtesting module in the latest version.

What BotSpot MCP adds

Use Claude, ChatGPT, Codex, Cursor, VS Code or another MCP-compatible assistant? Connect the free BotSpot MCP for:

Skills: build LumiBot strategies.
Free data: news, economics and SEC filings.
Backtests: run tests and read charts.
Cloud hosting: low-cost scheduled bots.
Community: share bots and discuss trades.
Marketplace: discover and publish strategies.
Broker linking: connect accounts through BotSpot.

Connect your AI assistant

For most situations, you will want to use the Polygon backtester or the Yahoo backtester instead, they are much easier to use and get started with. The Pandas backtester is intended for advanced users who have their own data and want to use it with Lumibot.

Pandas backtester is named after the python dataframe library because the user must provide a strictly formatted dataframe. You can use any csv, parquet, database data, etc that you wish, but Lumibot will only accept one format of dataframe.

Pandas backtester allows for intra-day and inter-day backtesting. Time frames for raw data are 1 minute and 1 day.

Additionally, with Pandas backtester, it is possible to backtest stocks, stock-like securities, futures contracts, crypto and FOREX.

Pandas backtester is the most flexible backtester in Lumibot, but it is also the most difficult to use. It is intended for advanced users who have their own data and want to use it with Lumibot.

Start by importing the Pandas backtester as follows:

from lumibot.backtesting import PandasDataBacktesting, BacktestingBroker

Next, create your Strategy class as you normally would. You can use any of the built-in indicators or create your own. You can also use any of the built-in order types or create your own.

from lumibot.strategies import Strategy

class MyStrategy(Strategy):
    def on_trading_iteration(self):
        # Do something here

Lumibot will start trading at 0000 hrs for the first date and up to 2359 hrs for the last. This is considered to be in the default time zone of Lumibot unless changed. This is America/New York (aka: EST)

Pandas backtester will receive a dataframe in the following format:

Index:
name: datetime
type: datetime64

Columns:
names: ['open', 'high', 'low', 'close', 'volume']
types: float

Your dataframe should look like this:

Example Dataframe¶
datetime	open	high	low	close	volume
2020-01-02 09:31:00	3237.00	3234.75	3235.25	3237.00	16808
2020-01-02 09:32:00	3237.00	3234.00	3237.00	3234.75	10439
2020-01-02 09:33:00	3235.50	3233.75	3234.50	3234.75	8203
…	…	…	…	…	…
2020-04-22 15:56:00	2800.75	2796.25	2800.75	2796.25	8272
2020-04-22 15:57:00	2796.50	2794.00	2796.25	2794.00	7440
2020-04-22 15:58:00	2794.75	2793.00	2794.25	2793.25	7569

Other formats for dataframes will not work.

You can download an example CSV using the yfinance library as follows:

import yfinance as yf

# Download minute data for the last 5 days for AAPL
data = yf.download("AAPL", period="5d", interval="1m")

# Save the data to a CSV file
data.to_csv("AAPL.csv")

The data objects will be collected in a dictionary called pandas_data using the asset as key and the data object as value. Subsequent assets + data can be added and then the dictionary can be passed into Lumibot for backtesting.

One of the important differences when using Pandas backtester is that you must use an Asset object for each data csv file loaded. You may not use a symbol as you might in Yahoo backtester.

For example, if you have a CSV file for AAPL, you must create an Asset object for AAPL and then pass that into the Data object.

from lumibot.entities import Asset

asset = Asset(
    symbol="AAPL",
    asset_type=Asset.AssetType.STOCK,
)

Next step will be to load the dataframe from csv.

import pandas as pd

# The names of the columns are important. Also important that all dates in the
# dataframe are time aware before going into lumibot.
df = pd.read_csv("AAPL.csv")

Third we make a data object for the asset. The data object must have at least the asset object, the dataframe, and the timestep. The timestep can be either minute or day. If you are using minute data, you must have a minute timestep. If you are using daily data, you must have a day timestep.

from lumibot.entities import Data

data = Data(
    asset,
    df,
    timestep="minute",
)

Next, we create or add to the dictionary that will be passed into Lumibot.

pandas_data = {
    asset: data
}

Finally, we can pass the pandas_data dictionary into Lumibot and run the backtest.

# Run the backtesting
trader = Trader(backtest=True)
data_source = PandasDataBacktesting(
    pandas_data=pandas_data,
    datetime_start=backtesting_start,
    datetime_end=backtesting_end,
)
broker = BacktestingBroker(data_source)
strat = MyStrategy(
    broker=broker,
    budget=100000,
)
trader.add_strategy(strat)
trader.run_all()

In Summary¶

Putting all of this together, and adding in budget and strategy information, the code would look like the following:

Getting the data would look something like this (using yfinance to download, but you can use any data source you wish):

import yfinance as yf

# Download minute data for the last 5 days for AAPL
data = yf.download("AAPL", period="5d", interval="1m")

# Save the data to a CSV file
data.to_csv("AAPL.csv")

Then, the strategy and backtesting code might look like this:

import pandas as pd
from lumibot.backtesting import BacktestingBroker, PandasDataBacktesting
from lumibot.entities import Asset, Data
from lumibot.strategies import Strategy

# A simple strategy that buys AAPL on the first day
class MyStrategy(Strategy):
    def on_trading_iteration(self):
        if self.first_iteration:
            order = self.create_order("AAPL", 100, "buy")
            self.submit_order(order)

# Read the data from the CSV file (in this example you must have a file named "AAPL.csv"
# in a folder named "data" in the same directory as this script)
df = pd.read_csv("AAPL.csv")
asset = Asset(symbol="AAPL", asset_type=Asset.AssetType.STOCK)

pandas_data = {
    asset: Data(asset, df, timestep="minute"),
}

backtesting_start = pandas_data[asset].datetime_start  # or datetime(2025, 1, 1)
backtesting_end = pandas_data[asset].datetime_end      # or datetime(2025, 1, 31)

# Run the backtest
result = MyStrategy.run_backtest(
    PandasDataBacktesting,
    backtesting_start,
    backtesting_end,
    pandas_data=pandas_data,
)

Options Expiration Settlement Policy¶

Pandas backtesting follows broker-style defaults for options expiration:

Equity/ETF options are physically settled at expiration. - Short in-the-money contracts are modeled as assigned. - Long in-the-money contracts are modeled as exercised when account constraints allow delivery.
Index options are cash settled at intrinsic value and exported as cash_settled.
Out-of-the-money contracts expire as expired.

These statuses are written to the trade artifacts so downstream systems can distinguish normal fills from expiration lifecycle events.

Optional: Environment Variables¶

If you prefer not to specify backtesting_start and backtesting_end in code, you can set the following environment variables, and LumiBot will automatically detect them:

Variable	Description	Example
IS_BACKTESTING	(Optional) Read only by startup code that checks it. It does not change a `backtest()` call into a broker run; see Backtest, paper, or live: choose the runner.	False
BACKTESTING_START	(Optional) Start date (YYYY-MM-DD).	2025-01-01
BACKTESTING_END	(Optional) End date (YYYY-MM-DD).	2025-01-31

Below is the full code that relies entirely on environment variables:

import pandas as pd
from lumibot.backtesting import BacktestingBroker, PandasDataBacktesting
from lumibot.entities import Asset, Data
from lumibot.strategies import Strategy

class MyStrategy(Strategy):
    def on_trading_iteration(self):
        if self.first_iteration:
            order = self.create_order("AAPL", 100, "buy")
            self.submit_order(order)

if __name__ == "__main__":
    df = pd.read_csv("AAPL.csv")
    asset = Asset(symbol="AAPL", asset_type=Asset.AssetType.STOCK)
    pandas_data = {
        asset: Data(asset, df, timestep="minute"),
    }

    # We do not specify any backtesting_start or backtesting_end here.
    # LumiBot will look for them in environment variables (BACKTESTING_START, BACKTESTING_END).
    result = MyStrategy.run_backtest(
        PandasDataBacktesting,
        pandas_data=pandas_data
    )

Frequently asked questions¶

How do I backtest with my own CSV data?¶

Load the file into a pandas DataFrame, wrap it in a Data object with its Asset and timestep, and put it in a pandas_data dictionary keyed by the asset. Then pass PandasDataBacktesting and pandas_data=pandas_data to run_backtest(), as in the full example above.

What columns does my DataFrame need?¶

It needs open, high, low, close and volume. Upper case names such as Close are changed to lower case for you. The index should be the date and time. If it is not, LumiBot looks for a column named datetime, date, timestamp or time and uses that as the index.

What timezone should my data use?¶

If your timestamps have no timezone, LumiBot treats them as America/New_York time. If they have a timezone, LumiBot converts them to America/New_York. You can also pass timezone=... to Data to set the timezone of naive timestamps.

Which timesteps does the Pandas backtester support?¶

Use timestep="minute" for minute bars or timestep="day" for daily bars. The Data class also accepts "hour". The timestep must match the bars in your file.

Which assets can I backtest with my own data?¶

Stocks and stock-like securities, futures, crypto and forex. You must create an Asset for each data set. For crypto, also pass a quote asset (for example USD) to Data, or LumiBot raises an error.

Can I load data from Parquet or a database instead of CSV?¶

Yes. Any source works as long as you end up with a DataFrame in the format above. CSV is just the most common starting point.

Can I select the Pandas backtester with BACKTESTING_DATA_SOURCE?¶

No. Pass PandasDataBacktesting in your code instead. If BACKTESTING_DATA_SOURCE is set to another source, it wins over the class in your code, so leave it unset (or set it to none) when you use your own data.

How are option expirations handled with my own data?¶

Equity and ETF options settle physically, index options settle in cash at intrinsic value, and out of the money contracts expire. The trade files mark these events as assigned, exercised, cash_settled or expired.