Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support of lz4 (de)compression on MS Windows #6891

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

Add support of lz4 (de)compression on MS Windows #6891

monetdb-team opened this issue Nov 30, 2020 · 0 comments
Labels
enhancement New feature or request SQL

Comments

@monetdb-team
Copy link

Date: 2020-06-11 15:41:24 +0200
From: Martin van Dinther <<martin.van.dinther>>
To: SQL devs <>
Version: 11.37.7 (Jun2020)

Last updated: 2020-11-24 11:37:23 +0100

Comment 27827

Date: 2020-06-11 15:41:24 +0200
From: Martin van Dinther <<martin.van.dinther>>

User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:74.0) Gecko/20100101 Firefox/74.0
Build Identifier:

Export of data to a file with extension .lz4 is currently not possible on Windows release. It returns an error.
Simularly files compressed with lz4 and extension .lz4 cannot be read as a decompressed csv/tsv file.

Add support for lz4 (de)compression on Windows by including liblz4.dll at installation (like done for libbz2.dll, zlib1.dll).
Sources and/or Binaries (lz4_win_v1_9_2.zip) kan be found at https://github.com/lz4/lz4/releases/tag/v1.9.2

Reproducible: Always

Steps to Reproduce:

COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.gz';
-- succeeds
COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.lz4';
-- returns error

CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.gz');
-- succeeds
CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.lz4');
-- returns error

Actual Results:

sql>COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.gz';
271 affected rows
sql>-- succeeds
sql>COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.lz4';
could not open file 'D:\MonetDB\Exports\tables.tsv.lz4': No such file or directory
sql>-- returns error
sql>
sql>CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.gz');
sql>-- succeeds
sql>CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.lz4');
GDK reported error: store_hot_snapshot: Failed to open D:\MonetDB\Backups\demodb.tar..tmp..lz4 for writing
sql>-- returns error
sql>

Expected Results:

sql>COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.gz';
271 affected rows
sql>-- succeeds
sql>COPY SELECT * FROM sys.tables ORDER BY name INTO R'D:\MonetDB\Exports\tables.tsv.lz4';
271 affected rows
sql>
sql>CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.gz');
sql>-- succeeds
sql>CALL sys.hot_snapshot(R'D:\MonetDB\Backups\demodb.tar.lz4');
sql>-- succeeds

Comment 28163

Date: 2020-10-14 18:07:36 +0200
From: Martin van Dinther <<martin.van.dinther>>

This has been successfully implemented in MonetDB v11.39.5 (Oct2020).

Related hg change: https//devmonetdborg/hg/MonetDB?cmd=changeset;node=fad7c1e666c8

@monetdb-team monetdb-team added enhancement New feature or request SQL labels Nov 30, 2020
@sjoerdmullender sjoerdmullender added this to the Ancient Release milestone Feb 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request SQL
Projects
None yet
Development

No branches or pull requests

2 participants