digitalmars.D.learn - iopipe code to count lines in gzipped file works with v 0.1.7 but
- Andrew (30/30) Aug 07 2020 Hi,
- Steven Schveighoffer (46/56) Aug 07 2020 As of iopipe v0.2.0, io is no longer a required dependency, it's
- Steven Schveighoffer (4/20) Aug 07 2020 ugh. This is an issue with iopipe specifying io version 0.2.x. I will
- Steven Schveighoffer (4/10) Aug 07 2020 OK, iopipe 0.2.2 is released, with no specific io dependency.
- Andrew (7/17) Aug 08 2020 Thank you for looking into this. I assumed the problem was at my
- Steven Schveighoffer (8/13) Aug 07 2020 BTW the safety improvements only change whether it compiles as @safe or ...
Hi, This code to count lines in a gzipped file exits with "Program exited with code -9" when run with the latest version of the library, I guess because I am doing unsafe things. Could someone tell me how to change it to make it work? The actual program I'm writing processes a file line by line, so ideally I'd like to keep the structure of open a file, then foreach over it. Thanks very much Andrew /+ dub.sdl: name "hello" dependency "iopipe" version="~>0.2.0" +/ import std.stdio; import std.typecons; import iopipe.textpipe; import iopipe.zip; import iopipe.bufpipe; import std.io : File = File; void main() { auto counter = 0; auto fileToRead = File("file.gz").refCounted.bufd.unzip(CompressionFormat.gzip); foreach (line; fileToRead.assumeText.byLineRange!false) { counter++; } writeln(counter); }
Aug 07 2020
On 8/7/20 9:40 AM, Andrew wrote:Hi, This code to count lines in a gzipped file exits with "Program exited with code -9" when run with the latest version of the library, I guess because I am doing unsafe things. Could someone tell me how to change it to make it work? The actual program I'm writing processes a file line by line, so ideally I'd like to keep the structure of open a file, then foreach over it. Thanks very muchAs of iopipe v0.2.0, io is no longer a required dependency, it's optional. So you must also add a dependency for io. I tried adding dependency "io" version="~>0.3.0" But it fails with: Got no configuration for dependency io ~>0.3.1 of hello ~master!? If I add dependency "io" version="*" it works. I think this is an issue with dub when using an inline recipe file, but I don't know? Note that in this simple example, the line count is stored in the line pipe, you can retreive the number of lines by accessing the `segments` member of the pipe (undocumented, I have to fix that). So my code looks like: --- /+ dub.sdl: name "hello" dependency "iopipe" version="~>0.2.0" dependency "io" version="*" +/ import std.stdio; //import std.typecons; // refCounted not safe import iopipe.textpipe; import iopipe.zip; import iopipe.bufpipe; import iopipe.refc; // refCounted that is safe import std.io : File = File; // just a note, I don't know why you are renaming here... void main() safe // yay safe! { auto counter = 0; auto fileToRead = File("file.gz").refCounted.bufd .unzip(CompressionFormat.gzip) .assumeText .byLine; fileToRead.process(); writeln(fileToRead.segments); } --- FYI, I noticed that in my simple test, this outputs one less than the actual lines. I'll have to look into *that* too. That dependency on writeln also irks me ;) I need to get working on that iopipe replacement for it... -Steve
Aug 07 2020
On 8/7/20 8:57 PM, Steven Schveighoffer wrote:I tried adding dependency "io" version="~>0.3.0" But it fails with: Got no configuration for dependency io ~>0.3.1 of hello ~master!? If I add dependency "io" version="*" it works. I think this is an issue with dub when using an inline recipe file, but I don't know?ugh. This is an issue with iopipe specifying io version 0.2.x. I will fix this. -Steve
Aug 07 2020
On 8/7/20 9:31 PM, Steven Schveighoffer wrote:On 8/7/20 8:57 PM, Steven Schveighoffer wrote:OK, iopipe 0.2.2 is released, with no specific io dependency. It should work with io 0.3.1 as a separate listed dependency. -SteveI think this is an issue with dub when using an inline recipe file, but I don't know?ugh. This is an issue with iopipe specifying io version 0.2.x. I will fix this.
Aug 07 2020
On Saturday, 8 August 2020 at 02:06:36 UTC, Steven Schveighoffer wrote:On 8/7/20 9:31 PM, Steven Schveighoffer wrote:Thank you for looking into this. I assumed the problem was at my end, as you have actually saved my bacon before, processing exactly the same file, the problem being that it was a blocked gzip file (ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.201305 2.genotypes.vcf.gz) I will raise an issue, as I think io is being imported and linked ok.On 8/7/20 8:57 PM, Steven Schveighoffer wrote:OK, iopipe 0.2.2 is released, with no specific io dependency. It should work with io 0.3.1 as a separate listed dependency. -SteveI think this is an issue with dub when using an inline recipe file, but I don't know?ugh. This is an issue with iopipe specifying io version 0.2.x. I will fix this.
Aug 08 2020
On 8/7/20 9:40 AM, Andrew wrote:Hi, This code to count lines in a gzipped file exits with "Program exited with code -9" when run with the latest version of the library, I guess because I am doing unsafe things.BTW the safety improvements only change whether it compiles as safe or not. If it's building but running is exiting with a code then it's possible there's a bug somewhere. I did have to change a lot of code to get it to build properly. If you have it building, but it's exiting with an error, then please file an issue with an example file and sample source that causes the issue. -Steve
Aug 07 2020