digitalmars.D.announce - ThePath - Convenient lib to deal with paths and files. (Alpha version)
- Dmytro Katyukha (91/91) Jan 15 2023 Hi,
- H. S. Teoh (39/49) Jan 17 2023 Yes it would be nice. But there may be security implications. For
- Dmytro Katyukha (3/16) Jan 19 2023 Hi,
Hi, I would like to anounce the new [ThePath](https://code.dlang.org/packages/thepath) published to https://code.dlang.org/. The basic functionality of [ThePath](https://code.dlang.org/packages/thepath) library seems to be completed, and before stabilizing it, i would like to ask community to review this lib and suggest what could be added / changed to make it usable for your usecases. Also, currently, this lib is tested only on linux, thus, may be some of you would be interested in making it cross-platform (help with testing and adding unittests for other platforms (Windows, MacOs, etc)). It should work on other platforms, but possibly with restricted functionality. Basically, this lib provides single struct `Path`, that have to be used to deal with file system paths and files in object-oriented way, making code more readable. Also, this lib contains function [createTempDirectory](https://github.com/katyukha/thepath/blob/master/sour e/thepath/utils.d), that, i think, would be nice to have it in Phobos. So, the questions are: - Do it have sense to convert `Path` to a class? Or keep it as struct? - Do it have sense to convert `Path` to template struct to make it possible to work with other types of strings (except `string` type)? - What are the requirements to place [createTempDirectory](https://github.com/katyukha/thepath/blob/master/source/ hepath/utils.d#L11) function in Phobos? - What else could be changed to make it better? Short list of features: - automatic expansion of `~` when needed (before passing path to std.file or std.stdio funcs) - single method to copy path (file or directory) to dest path - single method to remove path (file or directory) - simple method to `walk` through the path - `foreach(p; Path.current.walk) writeln(p.toString);` - `foreach(p; Path("/tmp").walk) writeln(p.toString);` - simple construction of paths from parts: - `Path("a", "b", "c")` - `Path("a").join("b", "c")` - simple deconstruction of paths - `Path("a/b/c/d").segments == ["a", "b", "c", "d"]` - `Path("a", "b", "c", "d").segments == ["a", "b", "c", "d"]` - overriden comparison operators for paths. - `Path("a", "b") == Path("a", "b")` - `Path("a", "b") != Path("a", "c")` - `Path("a", "b") < Path("a", "c")` - `hasAttributes` / `getAttributes` / `setAttributes` methods to work with file attrs - file operations as methods: - `Path("my-path").writeFile("Hello world")` - `Path("my-path").readFile()` - support search by glob-pattern - `foreach(path; Path.current.glob("*.py")) writeln(p.toString);` Short example is described below. More examples available in unittests and documentation. ```d import thepath; Path app_dir = Path("~/.local/my-app"); Path catalog_dir = app_dir.join("catalog"); void init() { // Note, that automatic '~' expansion will be done before checking the // existense of directory if (!app_dir.exists) { app_dir.mkdir(true); // create recursive } if (!catalog_dir.exists) { catalog_dir.mkdir(true); } } void list_dir() { // Easily print content of the catalog directory foreach(Path p; catalog_dir.walkBreadth) { writeln(p.toAbsolute().toString()); } } // Print all python files in current directory void find_python_files() { foreach(path; Path.current.glob("*.py", SpanMode.breadth)) // Print paths relative to current directory writeln(p.relativeTo(Path.current).toString); } Path findConfig() { // Search for "my-project.conf" in current directories and in // its parent directories auto config = Path.current.searchFileUp("my-project.conf"); enforce(!config.isNull); return config.get; } ```
Jan 15 2023
On Sun, Jan 15, 2023 at 01:53:51PM +0000, Dmytro Katyukha via Digitalmars-d-announce wrote: [...]Also, this lib contains function [createTempDirectory](https://github.com/katyukha/thepath/blob/master/source/thepath/utils.d), that, i think, would be nice to have it in Phobos.Yes it would be nice. But there may be security implications. For Posix, I see you use mkdtemp, which is secured by the OS / libc implementor. But for non-Posix, you used std.random; this is insecure because std.random is not intended for cryptographic applications, and anything not designed for crytographic security is vulnerable to exploits. Also, you need to be careful with the default permissions with the temp directory is created; leaving it up to whatever's set in the user's environment is generally unwise.So, the questions are: - Do it have sense to convert `Path` to a class? Or keep it as struct?Struct. In general, idiomatic D code prefers structs over classes. If you're not using inheritance and runtime polymorphism, there's no need to use classes.- Do it have sense to convert `Path` to template struct to make it possible to work with other types of strings (except `string` type)?IMO, this only introduces needless complexity. For example std.regex templatizes over char/wchar/dchar, but I've basically never needed to use anything except the char instantiation. This needless template parametrization only adds to std.regex's slow compile times; in retrospect it was IMO a mistake. Regular D code should just use strings (UTF-8) for everything, and convert to wstring at the OS boundary if you're on Windows and need something to be in UTF-16. And dstring is essentially useless; I've not heard of anyone needing to use dstring for the 10 or so years I've been using D. Just use string, that's good enough.- What are the requirements to place [createTempDirectory](https://github.com/katyukha/thepath/blob/master/source/thepath/utils.d#L11) function in Phobos?Use Phobos coding style, bring it up to Phobos coding standards.- What else could be changed to make it better?[...] Probably should always use the libc or OS function for creating a temp directory; it's generally bad idea to roll your own when it comes to creating temporary files or directories where there can be serious security implications. Other than insecure random name generation, there's also timing issues to be considered, i.e., if an attacker could predict the name, he could preemptively create the directory with the wrong permissions between your call to std.file.exists and std.file.mkdir, and exploit those permissions to manipulate the behaviour of your program later. You need to leverage OS APIs to guarantee the atomicity of checking for existence and creating the directory. T -- Ignorance is bliss... until you suffer the consequences!
Jan 17 2023
Dmytro Katyukha <firemage.dima gmail.com> On Tuesday, 17 January 2023 at 23:12:26 UTC, H. S. Teoh wrote:On Sun, Jan 15, 2023 at 01:53:51PM +0000, Dmytro Katyukha via Digitalmars-d-announce wrote: [...]Hi, Thank you for your feedback)[...]Yes it would be nice. But there may be security implications. For Posix, I see you use mkdtemp, which is secured by the OS / libc implementor. But for non-Posix, you used std.random; this is insecure because std.random is not intended for cryptographic applications, and anything not designed for crytographic security is vulnerable to exploits. Also, you need to be careful with the default permissions with the temp directory is created; leaving it up to whatever's set in the user's environment is generally unwise. [...]
Jan 19 2023