Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion integrate/storage/api/import-export.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The imported files must conform to the [RFC4180 Specification](https://tools.iet
![Schema of file upload process](/integrate/storage/api/async-import-handling.svg)

Exporting a table from Storage is analogous to its importing. First, data is [asynchronously
exported](https://keboola.docs.apiary.io/#reference/tables/unload-data-asynchronously/asynchronous-export) from
exported](https://api.keboola.com/?service=storage#post-/v2/storage/branch/-branchId-/tables/-id-/export-async) from
Table Storage into File Uploads. Then you can request to [download
the file](https://api.keboola.com/?service=storage#get-/v2/storage/branch/-branchId-/files/-fileId-), which will give you
access to an S3 server for the actual file download.
Expand Down
4 changes: 2 additions & 2 deletions integrate/storage/api/importer.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ curl --request POST --header "X-StorageApi-Token:storage-token" --form "tableId=
{% endhighlight %}

Using the Storage API Importer is the easiest way to upload data into Storage (except for
using one of the [API clients](/integrate/storage/#clients)). However, the disadvantage is that the whole data file
using one of the [API clients](/integrate/storage/#storage-api-clients)). However, the disadvantage is that the whole data file
has to be posted in a single HTTP request. **The maximum limit for a file size is 2GB and the transfer time is 45 minutes**.
This means that for substantially large files (usually more than hundreds of MB)
you may experience timeouts. If that happens, use the above outlined approach and upload the
Expand All @@ -34,7 +34,7 @@ files [directly to S3](/integrate/storage/api/import-export/#manually-uploading-
- `escapedBy` (optional) CSV escape character; empty by default.
- `incremental` (optional) If incremental is set to 0 (its default), the target table is truncated before each import.

Full list of avaialable parameters is available in the [API documentation](https://api.keboola.com/?service=import#import).
Full list of available parameters is available in the [API documentation](https://api.keboola.com/?service=import#import).

## Examples
To load data incrementally (append new data to existing contents):
Expand Down
2 changes: 1 addition & 1 deletion integrate/storage/docker-cli-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ or on Windows:

docker run --volume=C:\Users\name\some-dir:/data quay.io/keboola/storage-api-cli:latest create-table in.c-main new-table /data/new-table.csv --token=storage_token

or when using other then [default US region](/overview/api/#regions-and-endpoints), you need to provide the Storage API address:
or when using other than the [default US region](/overview/api/#regions-and-endpoints), you need to provide the Storage API address:

{% highlight bash %}
docker run --volume=$("pwd"):/data quay.io/keboola/storage-api-cli:latest create-table in.c-main new-table /data/new-table.csv --token=storage_token --url="https://connection.eu-central-1.keboola.com/"
Expand Down
7 changes: 6 additions & 1 deletion integrate/storage/php-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ use Keboola\StorageApi\Client;

$client = new Client([
'token' => 'your-token',
'url' => 'https://connection.keboola.com',
]);
{% endhighlight %}

Expand All @@ -88,6 +89,7 @@ use Keboola\StorageApi\Client;

$client = new Client([
'token' => 'your-token',
'url' => 'https://connection.keboola.com',
]);
$csvFile = new CsvFile('./new-table.csv');
$client->createTableAsync('in.c-main', 'new-table', $csvFile);
Expand All @@ -105,6 +107,7 @@ use Keboola\StorageApi\Client;

$client = new Client([
'token' => 'your-token',
'url' => 'https://connection.keboola.com',
]);
$csvFile = new CsvFile('./new-table.csv');
$client->writeTableAsync('in.c-main.new-table', $csvFile);
Expand All @@ -122,6 +125,7 @@ use Keboola\StorageApi\Client;

$client = new Client([
'token' => 'your-token',
'url' => 'https://connection.keboola.com',
]);
$csvFile = new CsvFile('./new-table.csv');
$client->writeTableAsync('in.c-main.new-table', $csvFile, ['incremental' => true]);
Expand All @@ -141,7 +145,8 @@ use Keboola\StorageApi\Client;
use Keboola\StorageApi\TableExporter;

$client = new Client([
'token' => 'your-token'
'token' => 'your-token',
'url' => 'https://connection.keboola.com',
]);

$exporter = new TableExporter($client);
Expand Down
14 changes: 9 additions & 5 deletions integrate/storage/r-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,12 @@ To list available commands, run
and the above command works.

The client is implemented as an [RC class](http://adv-r.had.co.nz/R5.html). To work with it, create an instance of the client.
The only required argument to create it is a valid Storage API token.
The required arguments to create it are a valid Storage API token and the API URL.

{% highlight r %}
client <- SapiClient$new(
token = 'your-token'
token = 'your-token',
url = 'https://connection.keboola.com'
)
{% endhighlight %}

Expand All @@ -60,7 +61,8 @@ To create the `new-table` table in the `in.c-main` bucket, use
{% highlight r %}
myDataFrame <- data.frame(id = c(1,2,3,4), secondCol = c('a', 'b', 'c', 'd'))
client <- SapiClient$new(
token = 'your-token'
token = 'your-token',
url = 'https://connection.keboola.com'
)

table <- client$saveTable(
Expand All @@ -82,7 +84,8 @@ To export data from the `old-table` table in the `in.c-main` bucket, use

{% highlight r %}
client <- SapiClient$new(
token = 'your-token'
token = 'your-token',
url = 'https://connection.keboola.com'
)

data <- client$importTable('in.c-main.old-table')
Expand All @@ -96,7 +99,8 @@ a [data.table](https://cran.r-project.org/web/packages/data.table/index.html) ob
{% highlight r %}
# create a client
client <- SapiClient$new(
token = 'your-token'
token = 'your-token',
url = 'https://connection.keboola.com'
)

# verify the token
Expand Down
Loading