Get Started
No framework information available.
README
Faborite π―
Sync Microsoft Fabric lakehouse data locally for faster development.
Faborite lets you pull sample data from your Fabric Lakehouses to your local machine, so you can develop and test notebooks/scripts without waiting for cloud compute.
Why Faborite?
When working with Microsoft Fabric, you often need to:
- π’ Wait for cloud compute to spin up just to test a simple query
- πΈ Pay for compute time during development iterations
- π Context-switch between local and cloud environments
Faborite solves this by bringing a representative sample of your data locally, enabling:
- β‘ Instant iteration - No cold start, no waiting
- π° Cost savings - Develop locally, deploy to cloud
- π§ͺ Better testing - Test with real data patterns locally
Features
- π² Smart Sampling - Random, recent, stratified, or custom SQL sampling
- π¦ Multiple Formats - Export to Parquet, CSV, JSON, or DuckDB
- β‘ Fast - Parallel downloads, DuckDB-powered sampling
- π§ Configurable - Sensible defaults, fully customizable per-table
- π Secure - Uses Azure authentication (CLI, Service Principal, Managed Identity)
- π Single Executable - Built with .NET 10 for fast startup and easy deployment
- π‘οΈ Production Ready - Comprehensive validation, logging, and retry policies
Installation
Download Binary
Download the latest release from GitHub Releases:
| Platform | Download |
|---|---|
| Windows (x64) | faborite-win-x64.zip |
| Linux (x64) | faborite-linux-x64.tar.gz |
| macOS (x64) | faborite-osx-x64.tar.gz |
| macOS (ARM64) | faborite-osx-arm64.tar.gz |
As .NET Global Tool
dotnet tool install -g faborite
From Source
git clone https://github.com/mjtpena/faborite.git
cd faborite
dotnet build
Quick Start
1. Login to Azure
az login
2. Initialize Configuration
faborite init
Edit faborite.json with your workspace and lakehouse IDs.
3. Sync Data
# Sync all tables with defaults (10,000 random rows each)
faborite sync --workspace <workspace-id> --lakehouse <lakehouse-id>
# Or use the config file
faborite sync
4. Use Your Data
import duckdb
df = duckdb.read_parquet('./local_lakehouse/customers/customers.parquet').df()
CLI Reference
sync
Sync data from OneLake to local machine.
faborite sync [options]
| Option | Short | Description | Default |
|---|---|---|---|
--workspace |
-w |
Workspace ID (GUID) | From config |
--lakehouse |
-l |
Lakehouse ID (GUID) | From config |
--config |
-c |
Config file path | faborite.json |
--rows |
-n |
Number of rows to sample | 10000 |
--strategy |
-s |
Sampling strategy | random |
--format |
-f |
Output format | parquet |
--output |
-o |
Output directory | ./local_lakehouse |
--table |
-t |
Tables to sync (repeatable) | All tables |
--skip |
Tables to skip (repeatable) | None | |
--parallel |
-p |
Max parallel downloads | 4 |
--no-schema |
Skip schema export | false |
Examples:
# Sync specific tables
faborite sync -w <id> -l <id> --table customers --table orders
# Custom sampling
faborite sync -w <id> -l <id> --rows 5000 --strategy recent
# Export as DuckDB database
faborite sync -w <id> -l <id> --format duckdb
# Export as CSV
faborite sync -w <id> -l <id> --format csv
list-tables (alias: ls)
List available tables in a lakehouse.
faborite list-tables -w <workspace-id> -l <lakehouse-id>
init
Generate a sample configuration file.
faborite init [options]
| Option | Short | Description | Default |
|---|---|---|---|
--output |
-o |
Output file path | faborite.json |
--force |
-f |
Overwrite existing file | false |
status
Show status of locally synced data.
faborite status [options]
| Option | Short | Description | Default |
|---|---|---|---|
--path |
-p |
Local data directory | ./local_lakehouse |
Sampling Strategies
| Strategy | Description | Use Case |
|---|---|---|
random |
Random sample using DuckDB's USING SAMPLE |
General development |
recent |
Most recent rows by date column | Time-series data |
head |
First N rows | Quick testing |
tail |
Last N rows | Recent additions |
stratified |
Proportional sample by column | Categorical data |
query |
Custom SQL query | Complex filters |
full |
All rows (no sampling) | Small lookup tables |
Configuration
Config File
Create a faborite.json file in your project root:
{
"workspaceId": "your-workspace-guid",
"lakehouseId": "your-lakehouse-guid",
"sample": {
"rows": 10000,
"strategy": "random"
},
"format": {
"output": "parquet",
"compression": "snappy"
},
"sync": {
"localPath": "./local_lakehouse",
"parallelTables": 4,
"includeSchema": true
},
"auth": {
"method": "cli"
},
"tableOverrides": {
"large_table": {
"rows": 1000,
"strategy": "recent",
"dateColumn": "created_at"
},
"lookup_table": {
"strategy": "full"
}
}
}
Environment Variables
All configuration can be overridden with environment variables:
| Variable | Description |
|---|---|
FABORITE_WORKSPACE_ID |
Workspace ID |
FABORITE_LAKEHOUSE_ID |
Lakehouse ID |
FABORITE_OUTPUT_PATH |
Output directory |
FABORITE_SAMPLE_ROWS |
Default sample rows |
FABORITE_FORMAT |
Output format |
AZURE_TENANT_ID |
Azure tenant for service principal auth |
AZURE_CLIENT_ID |
Azure client ID for service principal auth |
AZURE_CLIENT_SECRET |
Azure client secret for service principal auth |
Output Structure
./local_lakehouse/
βββ customers/
β βββ customers.parquet
β βββ _schema.json
βββ orders/
β βββ orders.parquet
β βββ _schema.json
βββ products/
β βββ products.parquet
β βββ _schema.json
βββ lakehouse.duckdb # When using --format duckdb
Authentication
Faborite uses Azure Identity for authentication:
| Method | Description | Config |
|---|---|---|
| Azure CLI (default) | Uses az login credentials |
"method": "cli" |
| Service Principal | App registration with secret | "method": "serviceprincipal" |
| Managed Identity | For Azure-hosted environments | "method": "managedidentity" |
| Default | Azure DefaultAzureCredential chain | "method": "default" |
Service Principal Setup
# Set environment variables
export AZURE_TENANT_ID="your-tenant-id"
export AZURE_CLIENT_ID="your-client-id"
export AZURE_CLIENT_SECRET="your-client-secret"
# Update config
{
"auth": {
"method": "serviceprincipal",
"tenantId": "your-tenant-id",
"clientId": "your-client-id"
}
}
Using with Notebooks
After syncing, load data in your local notebooks:
Python with DuckDB
import duckdb
# If exported as DuckDB
conn = duckdb.connect('./local_lakehouse/lakehouse.duckdb')
df = conn.execute("SELECT * FROM customers").df()
# If exported as Parquet
df = duckdb.read_parquet('./local_lakehouse/customers/customers.parquet').df()
Python with Pandas
import pandas as pd
df = pd.read_parquet('./local_lakehouse/customers/customers.parquet')
Python with Polars
import polars as pl
df = pl.read_parquet('./local_lakehouse/customers/customers.parquet')
.NET
using DuckDB.NET.Data;
using var connection = new DuckDBConnection("Data Source=./local_lakehouse/lakehouse.duckdb");
connection.Open();
// Query your data...
Requirements
- Runtime: .NET 10.0 (or use self-contained builds)
- Azure: Account with access to Microsoft Fabric
- Permissions: Read access to the target Lakehouse via OneLake
Finding Your IDs
- Workspace ID: Go to your Fabric workspace β Settings β Copy the Workspace ID
- Lakehouse ID: Open your Lakehouse β Settings β Copy the Lakehouse ID
Development
Prerequisites
- .NET 10.0 SDK
- Azure CLI (for authentication)
Building
# Clone the repository
git clone https://github.com/mjtpena/faborite.git
cd faborite
# Build
dotnet build
# Run tests
dotnet test
# Run with coverage
dotnet test --collect:"XPlat Code Coverage"
# Run the CLI
dotnet run --project src/Faborite.Cli -- sync -w <workspace-id> -l <lakehouse-id>
Publishing
# Publish self-contained for Windows
dotnet publish src/Faborite.Cli -c Release -r win-x64 --self-contained -o publish/win-x64
# Publish self-contained for Linux
dotnet publish src/Faborite.Cli -c Release -r linux-x64 --self-contained -o publish/linux-x64
# Publish self-contained for macOS
dotnet publish src/Faborite.Cli -c Release -r osx-x64 --self-contained -o publish/osx-x64
dotnet publish src/Faborite.Cli -c Release -r osx-arm64 --self-contained -o publish/osx-arm64
Project Structure
faborite/
βββ src/
β βββ Faborite.Core/ # Core library
β β βββ Configuration/ # Config loading & validation
β β βββ OneLake/ # OneLake ADLS Gen2 client
β β βββ Sampling/ # DuckDB-powered sampling
β β βββ Export/ # Format exporters
β β βββ Logging/ # Logging infrastructure
β β βββ Resilience/ # Retry policies (Polly)
β β βββ FaboriteService.cs # Main orchestrator
β βββ Faborite.Cli/ # CLI application
β βββ Commands/ # CLI commands
β βββ Program.cs # Entry point
βββ tests/
β βββ Faborite.Core.Tests/ # Core library tests
β βββ Faborite.Cli.Tests/ # CLI tests
βββ .github/
β βββ workflows/
β βββ ci.yml # CI/CD pipeline
βββ Faborite.sln
Roadmap
- Delta Lake time travel support
- Incremental sync (only changed data)
- Schema drift detection
- VS Code extension
- GitHub Action for CI/CD pipelines
- Support for Fabric Warehouses
Contributing
Contributions are welcome! Please read our Contributing Guide for details on our code of conduct and the process for submitting pull requests.
Quick Contribution Steps
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
Security
Please see our Security Policy for reporting vulnerabilities.
License
This project is licensed under the MIT License - see the LICENSE file for details.
Acknowledgments
- DuckDB - For blazing fast local analytics
- Azure SDK for .NET - For Azure integration
- Spectre.Console - For beautiful CLI output
- Polly - For resilience policies
Made with β€οΈ by Michael John PeΓ±a
Downloads by Version
Similar & Alternative Packages
Hand-picked matching NuGet libraries based on tags & ecosystem
Depended On By
Find packages that declare this package as a dependency. NuGet does not provide a reverse-dependency endpoint, so candidates are checked on demand.
Adoption Assessment
Assess this package against its supported frameworks, license, maintenance signals and dependency graph before making it a project standard.
Related Guides & Workflows
Learn how to evaluate, integrate, or audit this package in your .NET project.