Skip to content

Building from Source

Complete guide to building Rugo from source code.

Prerequisites

Required

  • Python: 3.9 or newer
  • C++ Compiler: C++17 compatible
  • Git: For cloning the repository

Compiler Requirements

Linux

# Debian/Ubuntu
sudo apt-get update
sudo apt-get install build-essential python3-dev

# Red Hat/Fedora
sudo dnf install gcc-c++ python3-devel

# Arch
sudo pacman -S base-devel python

Requires GCC 7+ or Clang 5+.

macOS

# Install Xcode Command Line Tools
xcode-select --install

Requires Clang from Xcode 10+.

Windows

Install Visual Studio 2017 or newer with C++ build tools:

  1. Download Visual Studio Build Tools
  2. Install "Desktop development with C++"
  3. Or install Build Tools for Visual Studio

Alternatively, use MinGW-w64.

Quick Start

# Clone repository
git clone https://github.com/mabel-dev/rugo.git
cd rugo

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install build dependencies and compile
make update
make compile

# Install in development mode
pip install -e .

Step-by-Step Build

1. Clone the Repository

git clone https://github.com/mabel-dev/rugo.git
cd rugo

2. Set Up Virtual Environment

# Create environment
python -m venv .venv

# Activate (Linux/macOS)
source .venv/bin/activate

# Activate (Windows)
.venv\Scripts\activate

3. Install Build Dependencies

Using the Makefile:

make update

Or manually:

pip install --upgrade pip setuptools wheel
pip install --upgrade cython
pip install -e ".[dev]"

4. Compile the Extension

Using the Makefile:

make compile

This: - Removes old build artifacts - Compiles Cython code - Builds C++ extension with -O3 optimization - Uses C++17 standard

Or manually:

# Clean old builds
rm -rf build/ rugo/parquet/*.cpp rugo/parquet/*.so

# Build extension
python setup.py build_ext --inplace

5. Verify Installation

import rugo.parquet as parquet_meta
print("Rugo installed successfully!")

Build Options

Debug Build

For development with debug symbols:

# Set environment variable
export CFLAGS="-g -O0"
export CXXFLAGS="-g -O0"

# Build
python setup.py build_ext --inplace

Release Build

For optimized production build:

# This is the default with 'make compile'
python setup.py build_ext --inplace -O3

Custom Compiler

Using Clang on Linux

export CC=clang
export CXX=clang++
make compile

Using Specific GCC Version

export CC=gcc-11
export CXX=g++-11
make compile

Development Builds

Install in Editable Mode

pip install -e .

This allows you to make changes without reinstalling.

Rebuild After Changes

After modifying C++ or Cython code:

make compile

Python-only changes don't require rebuilding.

Build Artifacts

Generated Files

build/                          # Build directory
rugo/parquet/metadata_reader.cpp  # Generated from .pyx
rugo/parquet/*.so               # Compiled extension (Linux)
rugo/parquet/*.dylib            # Compiled extension (macOS)
rugo/parquet/*.pyd              # Compiled extension (Windows)

Cleaning

Remove all build artifacts:

# Using Makefile
make clean

# Manual cleanup
rm -rf build/
rm -rf rugo.egg-info/
rm -rf rugo/parquet/*.so
rm -rf rugo/parquet/*.cpp
find . -name "__pycache__" -type d -exec rm -rf {} +
find . -name "*.pyc" -delete

Troubleshooting

Common Issues

Compiler Not Found

Error: error: command 'gcc' failed

Solution: Install build tools (see Prerequisites)

C++17 Not Supported

Error: error: unrecognized command line option '-std=c++17'

Solution: Update compiler to GCC 7+ or Clang 5+

Python.h Not Found

Error: fatal error: Python.h: No such file or directory

Solution: 

# Debian/Ubuntu
sudo apt-get install python3-dev

# Red Hat/Fedora
sudo dnf install python3-devel

Cython Import Error

Error: ModuleNotFoundError: No module named 'Cython'

Solution: 

pip install cython

Platform-Specific Issues

macOS - Invalid Architecture

Error: Building for wrong architecture

Solution: 

# For M1/M2 Macs
arch -arm64 python setup.py build_ext --inplace

# For Intel Macs
arch -x86_64 python setup.py build_ext --inplace

Windows - MSVC Not Found

Error: error: Microsoft Visual C++ 14.0 or greater is required

Solution: Install Visual Studio Build Tools

Linux - Missing Libraries

Error: Linking errors

Solution: 

sudo apt-get install python3-dev build-essential

Verification

Test the Build

# Run tests
make test

# Or manually
pytest

Check Extension

import rugo.parquet as parquet_meta

# Should work without errors
metadata = parquet_meta.read_metadata("test.parquet")

Verify Optimization

Check that extension was built with optimization:

# Linux/macOS
nm rugo/parquet/*.so | grep metadata

# Should show optimized symbols

Advanced Build Options

Custom Build Directory

python setup.py build_ext --build-lib=/custom/path

Parallel Compilation

python setup.py build_ext --parallel=4

Verbose Build

python setup.py build_ext --inplace --verbose

Continuous Integration

Example GitHub Actions workflow:

- name: Set up Python
  uses: actions/setup-python@v4
  with:
    python-version: 3.11

- name: Install dependencies
  run: |
    pip install --upgrade pip
    make update

- name: Compile
  run: make compile

- name: Test
  run: make test

Next Steps