Skip to content

Latest commit

 

History

6 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SnapSell — Local AI-Assisted Product Image Processor & Marketplace Listing Prototype

Status Frontend Backend Storage Database

SnapSell is a local end-to-end prototype for automating the preparation of product photos and draft marketplace listings.
It combines a React/Vite user interface, a NestJS API, S3-compatible object storage, Redis-based job queues, Dockerized background removal, post-processing with sharp, PostgreSQL persistence through Prisma, and CSV export preparation for marketplaces such as eBay and Etsy.

The project is designed around a practical e-commerce workflow:

take a raw product image → remove the background → generate marketplace-friendly image derivatives → save a draft listing → enrich listing metadata → export CSV-ready marketplace data.


Table of Contents

  1. Project Motivation
  2. Current Implementation Status
  3. Core Features
  4. System Architecture
  5. Technology Stack
  6. Image Processing Pipeline
  7. Marketplace Listing Workflow
  8. Repository Structure
  9. Prerequisites
  10. Quick Start on Windows
  11. Manual Startup
  12. Environment Configuration
  13. API Overview
  14. Database Model
  15. Production and Security Notes
  16. Troubleshooting
  17. Roadmap
  18. LinkedIn Project Summary
  19. License

Project Motivation

Creating marketplace listings is often a repetitive and time-consuming process.
A seller typically has to:

  • capture or collect product photos;
  • clean the image background;
  • create several aspect ratios required by different marketplaces;
  • prepare a title, description, price and attributes;
  • upload media assets;
  • format listing data for a target platform.

SnapSell explores how this workflow can be partially automated using a lightweight local architecture.
Instead of directly depending on external cloud services during development, the project uses local infrastructure components such as MinIO, Redis and PostgreSQL, while preserving a production-like system design.

The prototype is especially useful for learning and demonstrating:

  • asynchronous backend processing with queues;
  • signed URL based upload/download flows;
  • object-storage based media processing;
  • computer-vision assisted product image preparation;
  • marketplace-oriented listing data modeling;
  • full-stack integration between React, NestJS, Prisma and Dockerized processing tools.

Current Implementation Status

SnapSell is currently a local prototype, not a production SaaS platform.

Implemented in the repository:

  • React/Vite UI pages for image processing, draft listing list and draft editing.
  • NestJS API for upload URL generation, job enqueueing, job status polling, listing CRUD and CSV export.
  • BullMQ/Redis queue for asynchronous image processing.
  • MinIO/S3-compatible storage flow with presigned upload and download URLs.
  • Dockerized rembg execution for background removal.
  • Image derivative generation using sharp.
  • Prisma/PostgreSQL models for Listing and Asset.
  • PowerShell startup script for Windows-based local development.
  • Basic CSV export endpoint for eBay/Etsy-like workflows.

Not yet productionized:

  • real marketplace publishing through official APIs;
  • persistent OAuth account/token storage;
  • user authentication and authorization;
  • multi-user tenancy;
  • production Docker Compose for all services;
  • secure secrets management;
  • formal test coverage and CI/CD pipeline.

Core Features

1. Product Image Upload

The frontend requests a presigned PUT URL from the API, then uploads the local image file directly to MinIO/S3-compatible object storage.

This avoids routing large binary uploads through the application server and follows the same architectural principle used in scalable cloud media systems.

2. Asynchronous Processing

After upload completion, the UI calls the backend to enqueue a processing job.
The job is handled by BullMQ and Redis, allowing image processing to run outside the request/response path.

3. Background Removal

The worker executes rembg inside a Docker container.
The configured default model is isnet-general-use, with a fallback path to u2net if the first attempt fails.

4. Alpha Mask Post-Processing

The generated alpha mask is refined using configurable gain, blur and threshold parameters.
This step improves cutout stability and reduces artifacts around object boundaries.

Conceptually:

alpha_refined = threshold( blur( gain * alpha_original + offset ) )

5. Marketplace-Ready Derivatives

For each processed product image, SnapSell generates several output assets:

  • master transparent PNG cutout;
  • square transparent cutout;
  • square matte JPEG;
  • portrait 4:5 transparent cutout;
  • portrait 4:5 matte JPEG;
  • landscape 16:9 matte JPEG.

These formats are useful for different listing cards, product galleries, thumbnails and promotional layouts.

6. Draft Listing Management

Processed derivatives can be saved into a draft listing.
The draft editor supports:

  • title;
  • description;
  • price;
  • currency;
  • status;
  • cover image selection;
  • image ordering;
  • readiness checklist.

7. CSV Export Preparation

The API can export a listing as CSV-like marketplace data for eBay or Etsy style workflows.
The export includes basic metadata and ordered image URLs.


System Architecture

flowchart LR
    A[React / Vite UI] -->|POST /assets/upload-url| B[NestJS API]
    B -->|Presigned PUT URL| A
    A -->|PUT image| C[MinIO / S3 Upload Bucket]
    A -->|POST /assets/complete| B
    B -->|enqueue job| D[BullMQ Queue]
    D --> E[Redis]
    D --> F[Image Worker]
    F -->|download original| C
    F -->|run rembg in Docker| G[Background Removal Container]
    F -->|post-process with sharp| H[Derivative Generator]
    H -->|upload processed assets| I[MinIO / S3 Processed Bucket]
    A -->|GET /jobs/:id| B
    A -->|GET signed previews| B
    B -->|presigned GET URLs| I
    A -->|save/edit draft| B
    B --> J[(PostgreSQL / Prisma)]
    B -->|CSV export| K[eBay / Etsy CSV Preparation]
Loading

The architecture separates concerns into four major layers:

Layer Responsibility
Frontend User interaction, file selection, upload orchestration, status polling, draft editing
API Upload URL generation, queue orchestration, signed URL access, listing CRUD, CSV export
Worker Background removal, image normalization, alpha refinement, derivative generation
Infrastructure Redis queue backend, MinIO object storage, PostgreSQL database

Technology Stack

Frontend

  • React 18
  • Vite
  • React Router
  • Tailwind CSS
  • Axios / Fetch API
  • Local routes for processing and draft management

Backend

  • NestJS 11
  • TypeScript
  • BullMQ
  • ioredis
  • Prisma ORM
  • AWS SDK S3 client
  • sharp
  • Docker-driven rembg execution

Infrastructure

  • Redis 7 Alpine
  • MinIO S3-compatible storage
  • PostgreSQL 15+
  • Docker Desktop
  • Windows PowerShell startup script

Image Processing

  • rembg Docker container
  • isnet-general-use model by default
  • u2net fallback
  • Alpha-channel post-processing
  • Derivative rendering with sharp
  • JPEG matte outputs with soft shadow rendering

Image Processing Pipeline

The processing worker performs the following steps:

sequenceDiagram
    participant UI as React UI
    participant API as NestJS API
    participant Q as BullMQ / Redis
    participant W as Worker
    participant S3 as MinIO / S3
    participant R as rembg Docker
    participant DB as PostgreSQL

    UI->>API: Request presigned upload URL
    API-->>UI: Return storage_key and PUT URL
    UI->>S3: Upload original image
    UI->>API: Complete upload
    API->>Q: Enqueue process-asset job
    W->>S3: Download original
    W->>W: Normalize image
    W->>R: Remove background
    W->>W: Refine alpha mask
    W->>W: Generate derivatives
    W->>S3: Upload processed assets
    UI->>API: Poll job status
    API-->>UI: completed + derivative keys
    UI->>API: Save as draft
    API->>DB: Create Listing + Asset records
Loading

Processing Steps

  1. Download original image from the upload bucket.
  2. Normalize input to PNG and resize if the longest edge exceeds the configured limit.
  3. Run background removal through Dockerized rembg.
  4. Fallback model handling if the default model fails.
  5. Extract and refine alpha channel using gain, blur and threshold.
  6. Compute alpha bounding box to locate the product object.
  7. Expand and center crop to a target aspect ratio.
  8. Create a master transparent cutout.
  9. Generate derivative images for square, portrait and landscape layouts.
  10. Render matte JPEG versions with configurable background and soft shadow.
  11. Upload results to the processed bucket.
  12. Return derivative keys to the API/UI.

Marketplace Listing Workflow

flowchart TD
    A[Processed Product Images] --> B[Save as Draft]
    B --> C[Draft Listing]
    C --> D[Add title]
    C --> E[Add description]
    C --> F[Set price and currency]
    C --> G[Choose cover image]
    C --> H[Order image gallery]
    D --> I[Readiness Check]
    E --> I
    F --> I
    G --> I
    H --> I
    I -->|Ready| J[Export CSV]
    J --> K[eBay / Etsy Import Preparation]
Loading

A listing is considered ready when it has:

  • a non-empty title;
  • a positive price;
  • a currency value;
  • at least one matte image.

The current implementation supports status values such as:

draft | ready | published

At this stage, published is a logical state only. Real publishing through marketplace APIs is planned as a future step.


Repository Structure

snapsell/
├── app-ui/
│   ├── public/
│   ├── src/
│   │   ├── components/
│   │   ├── hooks/
│   │   ├── lib/
│   │   │   └── api.js
│   │   ├── pages/
│   │   │   ├── SnapProcessor.tsx
│   │   │   ├── DraftsList.jsx
│   │   │   └── DraftEditor.jsx
│   │   ├── Routes.jsx
│   │   └── index.jsx
│   ├── package.json
│   └── vite.config.mjs
│
├── backend/
│   └── api/
│       ├── prisma/
│       │   ├── schema.prisma
│       │   └── migrations/
│       ├── src/
│       │   ├── assets.controller.ts
│       │   ├── listings.controller.ts
│       │   ├── health.controller.ts
│       │   ├── image-utils.ts
│       │   ├── s3.service.ts
│       │   ├── prisma.service.ts
│       │   ├── jobs/
│       │   │   ├── jobs.module.ts
│       │   │   ├── jobs.service.ts
│       │   │   └── jobs.controller.ts
│       │   ├── app.module.ts
│       │   └── main.ts
│       └── package.json
│
├── infra/
│   └── rembg-cache/
│
├── start-snapsell.ps1
└── README.md

Note: the frontend currently also contains template pages from the initial UI scaffold. The SnapSell-specific pages are SnapProcessor.tsx, DraftsList.jsx and DraftEditor.jsx.


Prerequisites

The local prototype assumes the following environment:

  • Windows 10/11;
  • Docker Desktop;
  • Node.js 20+;
  • npm;
  • PostgreSQL 15+;
  • Git;
  • enough free disk space for Docker images, MinIO objects and background-removal model cache.

The rembg model is downloaded automatically during first execution inside the container.
Large binary model files are not committed to the repository.


Quick Start on Windows

From the repository root:

powershell -ExecutionPolicy Bypass -File .\start-snapsell.ps1

The script starts:

  • Redis container;
  • MinIO container;
  • API development server;
  • UI development server.

Expected local endpoints:

API:            http://localhost:8080
UI:             http://localhost:4028/processor
MinIO Console:  http://localhost:9001
MinIO S3 API:   http://localhost:9000

Default local MinIO credentials used by the development script:

user: minioadmin
pass: minioadmin

These credentials are intended only for local development.


Manual Startup

1. Redis

docker run -d --name snapsell-redis -p 6379:6379 redis:7-alpine --save "" --appendonly no

2. MinIO

docker run -d --name snapsell-minio `
  -p 9000:9000 `
  -p 9001:9001 `
  -e MINIO_ROOT_USER=minioadmin `
  -e MINIO_ROOT_PASSWORD=minioadmin `
  -v "$env:LOCALAPPDATA\snapsell\minio:/data" `
  quay.io/minio/minio server /data --console-address ":9001"

Create buckets:

docker run --rm --network host minio/mc alias set local http://localhost:9000 minioadmin minioadmin
docker run --rm --network host minio/mc mb --ignore-existing local/snapsell-user-uploads
docker run --rm --network host minio/mc mb --ignore-existing local/snapsell-processed
docker run --rm --network host minio/mc anonymous set download local/snapsell-processed

3. PostgreSQL

Create a local database, for example:

snapsell

Then configure DATABASE_URL in the backend environment file.

4. Backend API

cd backend/api
npm ci
npx prisma migrate dev --name init
npm run start:dev

5. Frontend UI

cd app-ui
npm ci
npm start

Environment Configuration

Backend: backend/api/.env

# S3 / MinIO
S3_ENDPOINT=http://localhost:9000
S3_REGION=us-east-1
S3_ACCESS_KEY=minioadmin
S3_SECRET_KEY=minioadmin
S3_BUCKET_UPLOADS=snapsell-user-uploads
S3_BUCKET_PROCESSED=snapsell-processed
S3_USE_PATH_STYLE=true

# Redis / BullMQ
REDIS_URL=redis://localhost:6379
JOB_QUEUE_PREFIX=snapsell

# Image processing
REMBG_ARGS=i -m isnet-general-use
REMBG_MAX_INPUT=2400
IMG_MAX_EDGE=2048
IMG_SHARPEN=0.8

DERIV_SQUARE=1200
DERIV_PORTRAIT_4X5=2000
DERIV_LAND_16X9=1920

JPEG_QUALITY=92
MATTE_BG=#ffffff

SHADOW_OPACITY=0.35
SHADOW_BLUR=35
SHADOW_OFFSET_Y=22
SHADOW_EXPAND=80

ALPHA_GAIN=1.6
ALPHA_OFFSET=0
ALPHA_BLUR=0.5
ALPHA_THRESH=40

# Optional rembg threading controls
OMP_NUM_THREADS=1
OPENBLAS_NUM_THREADS=1
MKL_NUM_THREADS=1

# Database
DATABASE_URL=postgresql://johndoe:secret@localhost:5432/snapsell?schema=public

# Public API base for CSV exports and signed URL generation
API_PUBLIC_BASE=http://localhost:8080

# Optional Etsy OAuth configuration
# ETSY_KEYSTRING=
# ETSY_SHARED_SECRET=
# ETSY_SCOPES=shops_r listings_r listings_w transactions_r
# ETSY_REDIRECT_URI=https://your-ngrok-domain.ngrok-free.dev/marketplaces/etsy/callback

Frontend: app-ui/.env

VITE_API_BASE=http://localhost:8080

API Overview

Health

GET /health

Returns a basic health status.

Assets

POST /assets/upload-url

Body:

{
  "filename": "product.jpg",
  "mime": "image/jpeg"
}

Returns:

{
  "storage_key": "uploads/...",
  "url": "presigned-put-url"
}
POST /assets/complete

Enqueues the background processing job.

{
  "storage_key": "uploads/...",
  "mime": "image/jpeg"
}
GET /assets/signed-get?key=processed/...

Returns a time-limited signed URL for processed assets.

Jobs

GET /jobs/:id

Returns queue state, result, error and timestamps.

Listings

POST /listings/save-draft
GET /listings?status=draft
GET /listings/:id
GET /listings/:id/ready
PATCH /listings/:id
GET /listings/:id/export?market=ebay
GET /listings/:id/export?market=etsy

The export endpoint returns a CSV file with listing metadata and ordered image URLs.


Database Model

The current Prisma schema centers around two core entities:

Listing

Represents a product listing draft or marketplace preparation record.

Main fields:

  • id
  • title
  • description
  • priceCents
  • currency
  • status
  • aspectsJSON
  • coverKey
  • createdAt
  • updatedAt
  • assets

Asset

Represents a processed image or media object associated with a listing.

Main fields:

  • id
  • bucket
  • key
  • mime
  • width
  • height
  • sortIndex
  • listingId

The database model is intentionally simple and suitable for extension with marketplace accounts, publishing states, category metadata, listing templates and audit logs.


Production and Security Notes

Before using this project beyond local experimentation, the following items should be addressed:

  • Replace local MinIO credentials with secure secrets.
  • Do not commit .env files containing real secrets.
  • Use HTTPS for all public endpoints.
  • Store OAuth tokens securely.
  • Add authentication and authorization.
  • Add rate limiting and request validation.
  • Limit maximum upload size and accepted MIME types.
  • Pin Docker image versions instead of relying on latest.
  • Isolate and resource-limit background processing containers.
  • Replace local object storage with production S3-compatible infrastructure or cloud storage.
  • Add structured logging, metrics and tracing.
  • Add automated tests for API endpoints, image pipeline and export formatting.
  • Add CI/CD validation.

Troubleshooting

Redis connection refused

Redis is probably not running. Start it manually or rerun the PowerShell startup script.

MinIO access denied

Check:

  • access key;
  • secret key;
  • bucket existence;
  • bucket policy;
  • endpoint URL;
  • path-style configuration.

rembg downloads the model repeatedly

Configure and mount a persistent model cache directory through REMBG_CACHE_DIR.

Image processing is slow

Background removal is CPU-intensive.
Performance depends on:

  • input image resolution;
  • selected model;
  • Docker runtime overhead;
  • CPU capability;
  • thread configuration.

For faster production processing, consider GPU-enabled inference, pre-warmed workers and model caching.

Job status disappears after completion

BullMQ jobs may be removed after completion depending on queue configuration.
The UI includes a fallback mechanism that attempts to infer completion by probing expected derivative keys.


Roadmap

Planned or recommended future improvements:

  • real Etsy/eBay API publishing;
  • OAuth account and token persistence;
  • listing templates by product category;
  • AI-assisted title and description generation;
  • automatic category suggestion;
  • bulk image upload and batch processing;
  • ZIP import/export;
  • drag-and-drop multi-image workflow;
  • background presets and shadow presets;
  • manual crop and mask correction;
  • production Docker Compose setup;
  • role-based admin UI;
  • observability dashboard;
  • unit, integration and e2e tests;
  • GitHub Actions pipeline;
  • marketplace-specific validation rules.

LinkedIn Project Summary

SnapSell is a full-stack local prototype for AI-assisted marketplace listing preparation.
The system automates a realistic e-commerce workflow: uploading a product image, removing the background with a Dockerized computer-vision model, generating marketplace-ready image derivatives, storing assets in S3-compatible storage, creating editable draft listings and exporting structured CSV data for eBay/Etsy-style workflows.

The project integrates React/Vite, NestJS, BullMQ, Redis, MinIO, PostgreSQL, Prisma, Docker, rembg and sharp, demonstrating a production-inspired architecture with asynchronous processing, signed URL media access and extensible listing data modeling.


License

No final license is currently defined in the repository.
For open-source publication, consider adding an MIT, Apache-2.0 or GPL-compatible license depending on the intended usage and contribution model.


SnapSell — локален AI-assisted процесор за продуктови изображения и прототип за маркетплейс листинги

Статус Frontend Backend Storage Database

SnapSell е локален end-to-end прототип за автоматизирана обработка на продуктови снимки и подготовка на чернови листинги за маркетплейси.
Проектът комбинира React/Vite потребителски интерфейс, NestJS API, S3-съвместимо object storage чрез MinIO, Redis/BullMQ опашки за асинхронна обработка, Dockerизиран rembg за премахване на фон, постпроцесинг чрез sharp, PostgreSQL база данни чрез Prisma и CSV експорт за eBay/Etsy-подобни workflows.

Основният практически поток е:

сурова продуктова снимка → премахване на фон → генериране на маркетплейс-ready изображения → запис като draft listing → редакция на метаданни → CSV експорт.


Съдържание

  1. Мотивация на проекта
  2. Текущ статус
  3. Основни функционалности
  4. Системна архитектура
  5. Технологичен стек
  6. Pipeline за обработка на изображения
  7. Workflow за маркетплейс листинг
  8. Структура на repository-то
  9. Предварителни изисквания
  10. Бърз старт под Windows
  11. Ръчно стартиране
  12. Environment конфигурация
  13. API overview
  14. Модел на базата данни
  15. Бележки за production и сигурност
  16. Отстраняване на проблеми
  17. Roadmap
  18. LinkedIn резюме
  19. Лиценз

Мотивация на проекта

Създаването на продуктови листинги в маркетплейси често е повтаряем и времеемък процес.
Обикновено продавачът трябва да:

  • заснеме или събере продуктови изображения;
  • премахне или изчисти фона;
  • подготви няколко aspect ratio формата;
  • напише заглавие, описание, цена и атрибути;
  • качи изображенията;
  • форматира данните за конкретен маркетплейс.

SnapSell изследва как този workflow може да бъде частично автоматизиран чрез лека локална архитектура.
Вместо в development фазата да зависи от външни cloud услуги, проектът използва локални инфраструктурни компоненти като MinIO, Redis и PostgreSQL, но запазва production-inspired системен дизайн.

Проектът е подходящ за демонстрация и обучение по:

  • асинхронна backend обработка чрез queue;
  • signed URL upload/download поток;
  • object-storage media processing;
  • computer-vision assisted продуктова фотография;
  • моделиране на marketplace listing данни;
  • full-stack интеграция между React, NestJS, Prisma и Docker.

Текущ статус

SnapSell е локален прототип, а не production SaaS платформа.

Реализирано в repository-то:

  • React/Vite UI страници за image processing, draft listing list и draft editor.
  • NestJS API за upload URL, enqueue на processing job, polling на job статус, CRUD за listings и CSV export.
  • BullMQ/Redis опашка за асинхронна обработка на изображения.
  • MinIO/S3-compatible storage поток с presigned upload и download URLs.
  • Dockerизиран rembg за премахване на фон.
  • Генериране на image derivatives чрез sharp.
  • Prisma/PostgreSQL модели за Listing и Asset.
  • PowerShell startup скрипт за Windows development среда.
  • Базов CSV export endpoint за eBay/Etsy-подобни workflows.

Все още не е production-ready:

  • реално публикуване чрез official marketplace APIs;
  • persistence на OAuth акаунти и tokens;
  • user authentication и authorization;
  • multi-user tenancy;
  • production Docker Compose за всички услуги;
  • secure secrets management;
  • формално test coverage и CI/CD pipeline.

Основни функционалности

1. Качване на продуктова снимка

Frontend-ът заявява presigned PUT URL от API-то, след което качва локалния image файл директно в MinIO/S3-compatible object storage.

Това избягва прехвърлянето на големи binary файлове през application server-а и следва архитектурен подход, използван в scalable cloud media системи.

2. Асинхронна обработка

След приключване на upload-а, UI извиква backend endpoint, който enqueue-ва processing job.
Job-ът се обработва чрез BullMQ и Redis, така че тежката image processing логика не блокира HTTP request/response потока.

3. Премахване на фон

Worker-ът стартира rembg вътре в Docker контейнер.
По подразбиране се използва модел isnet-general-use, като има fallback към u2net, ако първият опит се провали.

4. Постпроцесинг на alpha mask

Генерираната alpha маска се подобрява чрез конфигурируеми gain, blur и threshold параметри.
Целта е по-стабилен cutout и по-малко артефакти около границите на обекта.

Концептуално:

alpha_refined = threshold( blur( gain * alpha_original + offset ) )

5. Marketplace-ready derivatives

За всяка обработена снимка SnapSell генерира няколко изходни файла:

  • master transparent PNG cutout;
  • square transparent cutout;
  • square matte JPEG;
  • portrait 4:5 transparent cutout;
  • portrait 4:5 matte JPEG;
  • landscape 16:9 matte JPEG.

Тези формати са полезни за listing cards, product galleries, thumbnails и promotional layouts.

6. Управление на draft listings

Обработените derivatives могат да бъдат записани като draft listing.
Draft editor-ът поддържа:

  • заглавие;
  • описание;
  • цена;
  • валута;
  • статус;
  • избор на cover image;
  • подредба на изображения;
  • readiness checklist.

7. CSV export за маркетплейси

API-то може да експортира listing като CSV-подобни данни за eBay или Etsy workflow.
Експортът включва базови метаданни и подредени image URLs.


Системна архитектура

flowchart LR
    A[React / Vite UI] -->|POST /assets/upload-url| B[NestJS API]
    B -->|Presigned PUT URL| A
    A -->|PUT image| C[MinIO / S3 Upload Bucket]
    A -->|POST /assets/complete| B
    B -->|enqueue job| D[BullMQ Queue]
    D --> E[Redis]
    D --> F[Image Worker]
    F -->|download original| C
    F -->|run rembg in Docker| G[Background Removal Container]
    F -->|post-process with sharp| H[Derivative Generator]
    H -->|upload processed assets| I[MinIO / S3 Processed Bucket]
    A -->|GET /jobs/:id| B
    A -->|GET signed previews| B
    B -->|presigned GET URLs| I
    A -->|save/edit draft| B
    B --> J[(PostgreSQL / Prisma)]
    B -->|CSV export| K[eBay / Etsy CSV Preparation]
Loading

Архитектурата разделя системата на четири основни слоя:

Слой Отговорност
Frontend User interaction, избор на файл, upload orchestration, status polling, draft editing
API Генериране на upload URL, queue orchestration, signed URL access, listing CRUD, CSV export
Worker Background removal, image normalization, alpha refinement, derivative generation
Infrastructure Redis queue backend, MinIO object storage, PostgreSQL база данни

Технологичен стек

Frontend

  • React 18
  • Vite
  • React Router
  • Tailwind CSS
  • Axios / Fetch API
  • Локални routes за image processing и draft management

Backend

  • NestJS 11
  • TypeScript
  • BullMQ
  • ioredis
  • Prisma ORM
  • AWS SDK S3 client
  • sharp
  • Docker-driven rembg execution

Infrastructure

  • Redis 7 Alpine
  • MinIO S3-compatible storage
  • PostgreSQL 15+
  • Docker Desktop
  • Windows PowerShell startup script

Image Processing

  • rembg Docker container
  • isnet-general-use модел по подразбиране
  • u2net fallback
  • Alpha-channel post-processing
  • Derivative rendering чрез sharp
  • JPEG matte outputs с мека сянка

Pipeline за обработка на изображения

sequenceDiagram
    participant UI as React UI
    participant API as NestJS API
    participant Q as BullMQ / Redis
    participant W as Worker
    participant S3 as MinIO / S3
    participant R as rembg Docker
    participant DB as PostgreSQL

    UI->>API: Request presigned upload URL
    API-->>UI: Return storage_key and PUT URL
    UI->>S3: Upload original image
    UI->>API: Complete upload
    API->>Q: Enqueue process-asset job
    W->>S3: Download original
    W->>W: Normalize image
    W->>R: Remove background
    W->>W: Refine alpha mask
    W->>W: Generate derivatives
    W->>S3: Upload processed assets
    UI->>API: Poll job status
    API-->>UI: completed + derivative keys
    UI->>API: Save as draft
    API->>DB: Create Listing + Asset records
Loading

Processing steps

  1. Download на оригиналната снимка от upload bucket-а.
  2. Нормализация на входа към PNG и resize при твърде голям максимален размер.
  3. Background removal чрез Dockerизиран rembg.
  4. Fallback към втори модел, ако първият опит се провали.
  5. Извличане и refinement на alpha channel чрез gain, blur и threshold.
  6. Изчисляване на alpha bounding box за локализиране на продукта.
  7. Разширяване и центриране на crop-а към target aspect ratio.
  8. Генериране на master transparent cutout.
  9. Генериране на derivative изображения за square, portrait и landscape layouts.
  10. Рендериране на matte JPEG версии с фон и мека сянка.
  11. Upload на резултатите към processed bucket.
  12. Връщане на derivative keys към API/UI.

Workflow за маркетплейс листинг

flowchart TD
    A[Processed Product Images] --> B[Save as Draft]
    B --> C[Draft Listing]
    C --> D[Add title]
    C --> E[Add description]
    C --> F[Set price and currency]
    C --> G[Choose cover image]
    C --> H[Order image gallery]
    D --> I[Readiness Check]
    E --> I
    F --> I
    G --> I
    H --> I
    I -->|Ready| J[Export CSV]
    J --> K[eBay / Etsy Import Preparation]
Loading

Listing се счита за ready, когато има:

  • непразно заглавие;
  • положителна цена;
  • зададена валута;
  • поне едно matte изображение.

Текущо поддържаните статуси са:

draft | ready | published

На този етап published е логически статус. Реално публикуване чрез marketplace APIs е планирана бъдеща стъпка.


Структура на repository-то

snapsell/
├── app-ui/
│   ├── public/
│   ├── src/
│   │   ├── components/
│   │   ├── hooks/
│   │   ├── lib/
│   │   │   └── api.js
│   │   ├── pages/
│   │   │   ├── SnapProcessor.tsx
│   │   │   ├── DraftsList.jsx
│   │   │   └── DraftEditor.jsx
│   │   ├── Routes.jsx
│   │   └── index.jsx
│   ├── package.json
│   └── vite.config.mjs
│
├── backend/
│   └── api/
│       ├── prisma/
│       │   ├── schema.prisma
│       │   └── migrations/
│       ├── src/
│       │   ├── assets.controller.ts
│       │   ├── listings.controller.ts
│       │   ├── health.controller.ts
│       │   ├── image-utils.ts
│       │   ├── s3.service.ts
│       │   ├── prisma.service.ts
│       │   ├── jobs/
│       │   │   ├── jobs.module.ts
│       │   │   ├── jobs.service.ts
│       │   │   └── jobs.controller.ts
│       │   ├── app.module.ts
│       │   └── main.ts
│       └── package.json
│
├── infra/
│   └── rembg-cache/
│
├── start-snapsell.ps1
└── README.md

Забележка: frontend-ът все още съдържа template страници от първоначалния UI scaffold. SnapSell-специфичните страници са SnapProcessor.tsx, DraftsList.jsx и DraftEditor.jsx.


Предварителни изисквания

Локалният прототип предполага следната среда:

  • Windows 10/11;
  • Docker Desktop;
  • Node.js 20+;
  • npm;
  • PostgreSQL 15+;
  • Git;
  • достатъчно свободно дисково пространство за Docker images, MinIO обекти и model cache.

rembg моделът се изтегля автоматично при първо изпълнение вътре в контейнера.
Големи binary model файлове не са commit-нати в repository-то.


Бърз старт под Windows

От root директорията на проекта:

powershell -ExecutionPolicy Bypass -File .\start-snapsell.ps1

Скриптът стартира:

  • Redis container;
  • MinIO container;
  • API development server;
  • UI development server.

Очаквани локални endpoints:

API:            http://localhost:8080
UI:             http://localhost:4028/processor
MinIO Console:  http://localhost:9001
MinIO S3 API:   http://localhost:9000

Default локалните MinIO credentials в development скрипта са:

user: minioadmin
pass: minioadmin

Тези credentials са само за локална разработка.


Ръчно стартиране

1. Redis

docker run -d --name snapsell-redis -p 6379:6379 redis:7-alpine --save "" --appendonly no

2. MinIO

docker run -d --name snapsell-minio `
  -p 9000:9000 `
  -p 9001:9001 `
  -e MINIO_ROOT_USER=minioadmin `
  -e MINIO_ROOT_PASSWORD=minioadmin `
  -v "$env:LOCALAPPDATA\snapsell\minio:/data" `
  quay.io/minio/minio server /data --console-address ":9001"

Създаване на buckets:

docker run --rm --network host minio/mc alias set local http://localhost:9000 minioadmin minioadmin
docker run --rm --network host minio/mc mb --ignore-existing local/snapsell-user-uploads
docker run --rm --network host minio/mc mb --ignore-existing local/snapsell-processed
docker run --rm --network host minio/mc anonymous set download local/snapsell-processed

3. PostgreSQL

Създай локална база, например:

snapsell

След това конфигурирай DATABASE_URL в backend environment файла.

4. Backend API

cd backend/api
npm ci
npx prisma migrate dev --name init
npm run start:dev

5. Frontend UI

cd app-ui
npm ci
npm start

Environment конфигурация

Backend: backend/api/.env

# S3 / MinIO
S3_ENDPOINT=http://localhost:9000
S3_REGION=us-east-1
S3_ACCESS_KEY=minioadmin
S3_SECRET_KEY=minioadmin
S3_BUCKET_UPLOADS=snapsell-user-uploads
S3_BUCKET_PROCESSED=snapsell-processed
S3_USE_PATH_STYLE=true

# Redis / BullMQ
REDIS_URL=redis://localhost:6379
JOB_QUEUE_PREFIX=snapsell

# Image processing
REMBG_ARGS=i -m isnet-general-use
REMBG_MAX_INPUT=2400
IMG_MAX_EDGE=2048
IMG_SHARPEN=0.8

DERIV_SQUARE=1200
DERIV_PORTRAIT_4X5=2000
DERIV_LAND_16X9=1920

JPEG_QUALITY=92
MATTE_BG=#ffffff

SHADOW_OPACITY=0.35
SHADOW_BLUR=35
SHADOW_OFFSET_Y=22
SHADOW_EXPAND=80

ALPHA_GAIN=1.6
ALPHA_OFFSET=0
ALPHA_BLUR=0.5
ALPHA_THRESH=40

# Optional rembg threading controls
OMP_NUM_THREADS=1
OPENBLAS_NUM_THREADS=1
MKL_NUM_THREADS=1

# Database
DATABASE_URL=postgresql://johndoe:secret@localhost:5432/snapsell?schema=public

# Public API base за CSV exports и signed URL generation
API_PUBLIC_BASE=http://localhost:8080

# Optional Etsy OAuth configuration
# ETSY_KEYSTRING=
# ETSY_SHARED_SECRET=
# ETSY_SCOPES=shops_r listings_r listings_w transactions_r
# ETSY_REDIRECT_URI=https://your-ngrok-domain.ngrok-free.dev/marketplaces/etsy/callback

Frontend: app-ui/.env

VITE_API_BASE=http://localhost:8080

API overview

Health

GET /health

Връща базов health status.

Assets

POST /assets/upload-url

Body:

{
  "filename": "product.jpg",
  "mime": "image/jpeg"
}

Response:

{
  "storage_key": "uploads/...",
  "url": "presigned-put-url"
}
POST /assets/complete

Enqueue-ва background processing job.

{
  "storage_key": "uploads/...",
  "mime": "image/jpeg"
}
GET /assets/signed-get?key=processed/...

Връща time-limited signed URL за processed assets.

Jobs

GET /jobs/:id

Връща queue state, result, error и timestamps.

Listings

POST /listings/save-draft
GET /listings?status=draft
GET /listings/:id
GET /listings/:id/ready
PATCH /listings/:id
GET /listings/:id/export?market=ebay
GET /listings/:id/export?market=etsy

Export endpoint-ът връща CSV файл с listing metadata и подредени image URLs.


Модел на базата данни

Текущата Prisma схема е изградена около две основни entities:

Listing

Представлява draft или подготвен marketplace listing.

Основни полета:

  • id
  • title
  • description
  • priceCents
  • currency
  • status
  • aspectsJSON
  • coverKey
  • createdAt
  • updatedAt
  • assets

Asset

Представлява processed image/media обект, асоцииран с listing.

Основни полета:

  • id
  • bucket
  • key
  • mime
  • width
  • height
  • sortIndex
  • listingId

Моделът е умишлено изчистен и може лесно да се разшири с marketplace accounts, publishing states, category metadata, listing templates и audit logs.


Бележки за production и сигурност

Преди проектът да се използва извън локална експериментална среда, трябва да се добавят:

  • замяна на локалните MinIO credentials със secure secrets;
  • избягване на commit на .env файлове с реални secrets;
  • HTTPS за публични endpoints;
  • secure storage за OAuth tokens;
  • authentication и authorization;
  • rate limiting и request validation;
  • лимит на upload size и разрешени MIME types;
  • pinned Docker image versions вместо latest;
  • resource limits за background processing containers;
  • production S3-compatible storage или cloud object storage;
  • structured logging, metrics и tracing;
  • automated tests за API, image pipeline и export formatting;
  • CI/CD validation.

Отстраняване на проблеми

Redis connection refused

Вероятно Redis не работи. Стартирай го ръчно или пусни отново PowerShell startup скрипта.

MinIO access denied

Провери:

  • access key;
  • secret key;
  • дали buckets съществуват;
  • bucket policy;
  • endpoint URL;
  • path-style configuration.

rembg сваля модела при всяко стартиране

Конфигурирай и mount-ни persistent model cache directory чрез REMBG_CACHE_DIR.

Image processing-ът е бавен

Background removal е CPU-интензивна операция.
Производителността зависи от:

  • resolution на входното изображение;
  • избрания model;
  • Docker runtime overhead;
  • CPU capability;
  • thread configuration.

За production може да се разгледа GPU-enabled inference, pre-warmed workers и model caching.

Job status изчезва след completion

BullMQ jobs могат да бъдат премахнати след completion според queue конфигурацията.
UI-то има fallback механизъм, който опитва да infer-не completion чрез expected derivative keys.


Roadmap

Планирани или препоръчителни подобрения:

  • реално Etsy/eBay API publishing;
  • OAuth account и token persistence;
  • listing templates по product category;
  • AI-assisted title и description generation;
  • automatic category suggestion;
  • bulk image upload и batch processing;
  • ZIP import/export;
  • drag-and-drop multi-image workflow;
  • background presets и shadow presets;
  • manual crop и mask correction;
  • production Docker Compose setup;
  • role-based admin UI;
  • observability dashboard;
  • unit, integration и e2e tests;
  • GitHub Actions pipeline;
  • marketplace-specific validation rules.

LinkedIn резюме

SnapSell е full-stack локален прототип за AI-assisted подготовка на marketplace listings.
Системата автоматизира реалистичен e-commerce workflow: качване на продуктова снимка, премахване на фон чрез Dockerизиран computer-vision модел, генериране на marketplace-ready derivatives, съхранение в S3-compatible object storage, създаване на editable draft listings и export на структурирани CSV данни за eBay/Etsy-подобни процеси.

Проектът интегрира React/Vite, NestJS, BullMQ, Redis, MinIO, PostgreSQL, Prisma, Docker, rembg и sharp, демонстрирайки production-inspired архитектура с асинхронна обработка, signed URL media access и разширяем data model за listings.


Лиценз

В repository-то все още няма финално дефиниран лиценз.
За open-source публикуване може да се добави MIT, Apache-2.0 или GPL-compatible лиценз според желания модел на използване и принос.

About

Local AI-assisted marketplace-listing prototype with React, NestJS, MinIO/S3 uploads, Redis/BullMQ jobs, image background removal, PostgreSQL/Prisma and CSV export.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages