4.1 KiB

Raw Blame History

Millelibri project

This is a fork from zlib-searcher project. Future goals:

improve search indexes (language)
add books
expand file types

zlib(libgen) searcher

Search zlib/libgen index to get ipfs_cid.

We don't save and provide files, we provide search.

I hope everyone have a copy of the index locally, so that no need to rely on any centralized service.

Deploy with Docker

git clone https://github.com/zlib-searcher/zlib-searcher.git && cd zlib-searcher
wget https://github.com/zlib-searcher/zlib-searcher/releases/download/0.6.0/index_0.6.zip && unzip index_0.6.zip
docker-compose up -d

Now zlib-searcher it will listen to 0.0.0.0:7070.

Usage

1. Download the pre-compiled binary from Release.

Or you can compile by yourself. Refer to Build from source for instructions.

2. Download the `index` file that has been created.

We will give the corresponding index download links for each version in the release page.

Or you can make your own via zlib-searcher index.

Extract the index folder to the same level as the program, it should look like the following:

zlib_searcher_dir
├── index
│   ├── some index files...
│   └── meta.json
└── zlib-searcher

3. Run `zlib-searcher run`, it will listen to `127.0.0.1:7070`.

Access http://127.0.0.1:7070/ to use webui, or you can use the original api.

original search api

You can search by the following fields:

title
author
publisher
extension
language
isbn
zlib_id

Examples:

http://127.0.0.1:7070/search?limit=30&query=余华
http://127.0.0.1:7070/search?limit=30&query=title:机器学习 extension:azw3 publisher:清华
http://127.0.0.1:7070/search?limit=30&query=zlib_id:18557063
http://127.0.0.1:7070/search?limit=30&query=isbn:9787302423287

Build from source

1. Build `zlib-searcher`

First build frontend

make frontend_preinstall frontend

Then build zlib-searcher

TARGET=release make

# move the compiled binary to the project root directory
mv target/release/zlib-searcher .

2. Build `index`

Download zlib_index_books.csv.zip and libgen_index_books.csv.zip and extract the csv files to the project root directory.

Then run zlib-searcher index. You may need to rm index/* first.

If you have other csv files, you can run zlib-searcher index -f *.csv to index them.

The finally folder structure should look like this:

zlib_searcher_dir // in the example above, it is project root directory.
├── index
│   ├── some index files...
│   └── meta.json
└── zlib-searcher

Raw data

We downloaded libgen sql and zlib sql and exported the necessary data from them.

id, title, author, publisher, extension, filesize, language, year, pages, isbn, ipfs_cid

This raw data is used to generate our index, you can download the raw data from here:

4.1 KiB Raw Blame History