First steps

To start using the application you need to install PHP 7 with extensions listed in composer.json file (see entries ext-{name}), composer, and a database engine - SQLite3 or MySQL server.

You can also use a ready-to-use docker container instead of using host installation of PHP, if you have a possibility always use a docker container.

Summary of application requirements:

  • PHP7.2 or newer
  • SQLite3 or MySQL database
  • Composer (PHP package manager, see packagist.org)
  • make (GNU Make)

Manual installation

At first you need to create your own customized .env file with application configuration. You can create it from a template .env.dist.

Make sure the APP_ENV is set to prod.

cp .env .env.dist
edit .env

To install the application - download dependencies, install database schema use the make task install.

make install

All right! The application should be ready to go. To check the application you can launch a development web server.

make run_dev

Installation with docker

You have at least three choices:

  • Use wolnosciowiec/file-repository container by your own (advanced)
  • Use a prepared docker-compose environment placed in examples/docker directory
  • Create your own environment based on docker-compose

Proposed way to choose is the prepared docker-compose environment that is placed in examples/docker directory. Here are instructions how to start with it:

# go to the environment directory and copy template file
cd ./examples/docker
cp .env.dist .env

# adjust application settings
edit .env

Now adjust the environment variables to your need - you might want to see the configuration reference. If you think the configuration is finished, start the environment. To stop it - type CTRL+C.

# start the environment
make start

Example docker-compose.yml file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
version: '2'
    services:
        #
        # Optional: You can use AWS S3, Minio, but actually you don't have to.
        #           "local" adapter may be also used, then files would be stored locally on the disk
        #
        #           S3 gives possibility to replicate the data and/or redistribute it, relocate
        #
        app_minio:
            image: minio/minio
            volumes:
                - ./data/minio:/data
            expose:
                - "9000"
            command: server /data
            restart: on-failure
            mem_limit: 367001600 # 350M
            environment:
                - MINIO_DOMAIN=app_minio
                - MINIO_ACCESS_KEY=hello
                - MINIO_SECRET_KEY=hello123

        #
        # File Repository API application
        #
        app_storage:
            image: wolnosciowiec/file-repository:latest
            environment:
                - DATABASE_URL=mysql://file_repository:[email protected]_mysql:3306/file_repository
                - DATABASE_DRIVER=pdo_mysql

                # SQLite3 example
                #- DATABASE_URL=sqlite:///%kernel.project_dir%/var/data.db
                #- DATABASE_DRIVER=pdo_sqlite

                # minio adapter setup
                - FS_AWSS3V3_CLIENT=s3_client
                - FS_AWSS3V3_BUCKET=misc
                - FS_AWSS3V3_PREFIX=
                - FS_AWSS3V3_OPTIONS_ENDPOINT=http://app_minio:9000
                - FS_AWSS3V3_VERSION=latest
                - FS_AWSS3V3_KEY=hello
                - FS_AWSS3V3_SECRET=hello123
                - BACKUP_ONE_VERSION_MAX_SIZE=25GB
                - BACKUP_COLLECTION_MAX_SIZE=150GB
                - WAIT_FOR_HOST=db_mysql:3306

                # local adapter setup:
                #- FS_ADAPTER=local
                #- FS_LOCAL_DIRECTORY=%kernel.root_dir%/../var/uploads
            env_file:
                - .env
            expose:
                - "80"
            depends_on:
                - db_mysql
            restart: on-failure
            mem_limit: 524288000 # 500M

        #
        # Optional: A SQLite3 can be used instead MySQL
        #           MySQL gives a possibility to replicate the data
        #
        db_mysql:
            image: mysql:8
            environment:
                - MYSQL_USER=file_repository
                - MYSQL_PASSWORD=playground
                - MYSQL_DATABASE=file_repository
                - MYSQL_ROOT_PASSWORD=root
            volumes:
                - "./data/mysql:/var/lib/mysql"
            command: mysqld --innodb_file_per_table=1

        #
        # Test: Our test application we will make a backup for
        #       Access it via localhost:8100, modify it's files in ./www
        #
        app_webserver:
            image: nginx
            volumes:
                - "./www:/usr/share/nginx/html"
            ports:
                - "8100:80"

        #
        # Test: Backup client, normally it should be on a separate machine.
        #       For testing we have it in same docker network.
        #
        backup_client:
            image: wolnosciowiec/file-repository:bahub
            volumes:
                - "./data/backup/cron:/cron:ro"
                - "./data/backup/bahub.conf.yaml:/bahub.conf.yaml:ro"
                - "/var/run/docker.sock:/var/run/docker.sock"

Post-installation

At this point you have the application, but you do not have access to it. You will need to generate an administrative access token to be able to create new tokens, manage backups, upload files to storage. To achieve this goal you need to execute a simple command.

Given you use docker you can do eg. sudo docker exec some-container-name ./bin/console auth:generate-admin-token, for bare metal installation it would be just ./bin/console auth:generate-admin-token in the project directory.

So, when you have an administrative token, then you need a token to upload backups. It’s not recommended to use administrative token on your servers. Recommended way is to generate a separate token, that is allowed to upload a backup to specified collection

To do so, check all available roles in the application:

GET /auth/roles?_token=YOUR-ADMIN-TOKEN-HERE

Note: If you DO NOT KNOW HOW to perform a request, then please check the postman section

You should see something like this:

{
    "roles": {
        "upload.images": "Allows to upload images",
        "upload.documents": "Allows to upload documents",
        "upload.backup": "Allows to submit backups",
        "upload.all": "Allows to upload ALL types of files regardless of mime type",
        "security.authentication_lookup": "User can check information about ANY token",
        "security.overwrite": "User can overwrite files",
        "security.generate_tokens": "User can generate tokens with ANY roles",
        "security.use_technical_endpoints": "User can use technical endpoints to manage the application",
        "deletion.all_files_including_protected_and_unprotected": "Delete files that do not have a password, and password protected without a password",
        "view.any_file": "Allows to download ANY file, even if a file is password protected",
        "view.files_from_all_tags": "List files from ANY tag that was requested, else the user can list only files by tags allowed in token",
        "view.can_use_listing_endpoint_at_all": "Define that the user can use the listing endpoint (basic usage)",
        "collections.create_new": "Allow person creating a new backup collection",
        "collections.allow_infinite_limits": "Allow creating backup collections that have no limits on size and length",
        "collections.modify_any_collection_regardless_if_token_was_allowed_by_collection": "Allow to modify ALL collections. Collection don't have to allow such token which has this role",
        "collections.view_all_collections": "Allow to browse any collection regardless of if the user token was allowed by it or not",
        "collections.can_use_listing_endpoint": "Can use an endpoint that will allow to browse and search collections?",
        "collections.manage_tokens_in_allowed_collections": "Manage tokens in the collections where our current token is already added as allowed",
        "collections.upload_to_allowed_collections": "Upload to allowed collections",
        "collections.list_versions_for_allowed_collections": "List versions for collections where the token was added as allowed",
        "collections.delete_versions_for_allowed_collections": "Delete versions only from collections where the token was added as allowed"
    }
}

To allow only uploading and browsing versions for assigned collections you may choose:

POST /auth/token/generate?_token=YOUR-ADMIN-TOKEN-THERE
{
    "roles": ["upload.backup", "collections.upload_to_allowed_collections", "collections.list_versions_for_allowed_collections"],
    "data": {
        "tags": [],
        "allowedMimeTypes": [],
        "maxAllowedFileSize": 0
    }
}

As the response you should get the token id that you need.

{
    "tokenId": "34A77B0D-8E6F-40EF-8E70-C73A3F2B3AF8",
    "expires": null
}

Remember the tokenId, now you can create collections and grant access for this token to your collections. Generated token will be able to upload to collections you allow it to.

Check next steps:

  1. Collection creation
  2. Assigning a token to the collection

That’s all.