www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Standardpaths library

reply "FreeSlave" <freeslave93 gmail.com> writes:
I wrote small library for getting standard paths (like Pictures, 
Music)
Here's dub package http://code.dlang.org/packages/standardpaths
And github repo https://github.com/MyLittleRobo/standardpaths

You can see open issues on github. Please, participate in 
discussions if you're interested. The biggest problem now is OS X 
support https://github.com/MyLittleRobo/standardpaths/issues/4

You can generate documentation with dub build --build=docs.
Apr 05 2015
next sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 05 Apr 2015 09:08:12 +0000
schrieb "FreeSlave" <freeslave93 gmail.com>:


 I wrote small library for getting standard paths (like Pictures, 
 Music)
 Here's dub package http://code.dlang.org/packages/standardpaths
 And github repo https://github.com/MyLittleRobo/standardpaths
 
 You can see open issues on github. Please, participate in 
 discussions if you're interested. The biggest problem now is OS X 
 support https://github.com/MyLittleRobo/standardpaths/issues/4
 
 You can generate documentation with dub build --build=docs.


This is indeed useful. I once had a specific use case where I
needed the default location for where the assets of my
application is stored. On Windows applications come with an
installer and install the assets right into the installation
directory (usually current directory). On Linux applications
store their data in /usr/share/<appname>/. So with a
compile-time only template I accessed these as:

string asset1 = dirs!"myapp".staticData ~ "/image.png";

Now I realize this is a bit simplistic for several reasons:

* On Windows, one might still want to create a subdirectory to
  separate assets from program code, e.g. "data".

* On Linux, as your library shows, data directories are
  layered like this:
  ~/.local/share, /usr/local/share, /usr/share

* The _actual_ data directory is often passed into the
  configure script on Linux and no general algorithm could
  guess it.

* An application name would typically become lower-case on
  Linux while keeping its casing on Windows in e.g.
  AppData/<appname>.

Unsure if that's it, but that could make data file lookup for
application "SomeTool" go like this:

was the data directory explicitly overridden (e.g.
through ./configure or by the programmer)?
  return overridden_dir;  // note: relative directories expand
as based on executable directory
is this Linux?
  for (dir in
  ["${XDG_DATA_HOME}"/sometool, /usr/local/share/sometool, /usr/share/sometool])
    is the file in dir ?
      return dir
is this Windows?
  return executable_path

This goes a bit into heuristics and best practices, so it
probably doesn't fit with your library that provides
clearly defined standard paths from the desktop environment.

Aside from that I think it is a common enough use case to
lookup assets that ship with your program in the "typical"
installation directories.


On another note when I ran your 'printdirs' it didn't list a
user Fonts or Applications directory. The Applications
directory is ok, but I do have a ~/.fonts/ directory and
/etc/fonts/fonts.conf says:
  <!-- the following element will be removed in the future -->
  <dir>~/.fonts</dir>
Fonts in ~/.fonts are listed in LibreOffice. It seems like you
do parse /etc/fonts/fonts.conf. Maybe there is a bug in the
parser?

The whole Applications thing doesn't make much sense on Linux,
right? Is that a directory where applications are installed to
including their assets?

-- 
Marco
Apr 05 2015
next sibling parent reply "FreeSlave" <freeslave93 gmail.com> writes:
On Sunday, 5 April 2015 at 11:42:42 UTC, Marco Leise wrote:

 On another note when I ran your 'printdirs' it didn't list a
 user Fonts or Applications directory. The Applications
 directory is ok, but I do have a ~/.fonts/ directory and
 /etc/fonts/fonts.conf says:
   <!-- the following element will be removed in the future -->
   <dir>~/.fonts</dir>
 Fonts in ~/.fonts are listed in LibreOffice. It seems like you
 do parse /etc/fonts/fonts.conf. Maybe there is a bug in the
 parser?

 The whole Applications thing doesn't make much sense on Linux,
 right? Is that a directory where applications are installed to
 including their assets?


Probably you don't have local 
$XDG_CONFIG_DIR/fontconfig/fonts.conf file.

I've opened issue 
https://github.com/MyLittleRobo/standardpaths/issues/8

About Applications: on my Windows 7 it returns 
C:/Users/Username/Application Data/Microsoft/Windows/Start 
Menu/ProgramsC:/ProgramData/Microsoft/Windows/Start Menu/Programs 
where .lnk files are stored (I believe these are used in the 
start  menu). Since freedesktop systems use .desktop files it 
would be sane to return paths which contain them 
(~/.local/share/applications, /usr/local/share/applications and 
/usr/share/applications). I just have not implemented it yet. 
Though not sure it the whole thing can be useful, since things 
are not the same on Windows and freedesktop: Windows uses 
directories to make menu hierarchy, while freedesktop for the 
same purpose use Categories field in .desktop files. Also .lnk 
and .desktop are different things themselves.
Apr 05 2015
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 05 Apr 2015 12:31:23 +0000
schrieb "FreeSlave" <freeslave93 gmail.com>:


 On Sunday, 5 April 2015 at 11:42:42 UTC, Marco Leise wrote:

 On another note when I ran your 'printdirs' it didn't list a
 user Fonts or Applications directory. The Applications
 directory is ok, but I do have a ~/.fonts/ directory and
 /etc/fonts/fonts.conf says:
   <!-- the following element will be removed in the future -->
   <dir>~/.fonts</dir>
 Fonts in ~/.fonts are listed in LibreOffice. It seems like you
 do parse /etc/fonts/fonts.conf. Maybe there is a bug in the
 parser?

 The whole Applications thing doesn't make much sense on Linux,
 right? Is that a directory where applications are installed to
 including their assets?


 
 Probably you don't have local 
 $XDG_CONFIG_DIR/fontconfig/fonts.conf file.
 
 I've opened issue 
 https://github.com/MyLittleRobo/standardpaths/issues/8
 
 About Applications: on my Windows 7 it returns 
 C:/Users/Username/Application Data/Microsoft/Windows/Start 
 Menu/ProgramsC:/ProgramData/Microsoft/Windows/Start Menu/Programs 
 where .lnk files are stored (I believe these are used in the 
 start  menu). Since freedesktop systems use .desktop files it 
 would be sane to return paths which contain them 
 (~/.local/share/applications, /usr/local/share/applications and 
 /usr/share/applications). I just have not implemented it yet. 
 Though not sure it the whole thing can be useful, since things 
 are not the same on Windows and freedesktop: Windows uses 
 directories to make menu hierarchy, while freedesktop for the 
 same purpose use Categories field in .desktop files. Also .lnk 
 and .desktop are different things themselves.


You are right, the two are very different. One has to write OS
specific code to use them. The funny thing is, D as a systems
programming language could actually be used by someone to
write a Linux package manager or Windows installer. :p

Do as you see fit. Qt as an inspiration is a good thing I
believe. Some classes I had a look at were intuitive and well
thought out.

-- 
Marco
Apr 06 2015
prev sibling parent reply "FreeSlave" <freeslave93 gmail.com> writes:
On Sunday, 5 April 2015 at 11:42:42 UTC, Marco Leise wrote:

is this Windows?
 return executable_path


That depends on what do you understand by data. Are game's saves 
data too? Or content downloaded while playing (server-specific 
assets or new levels). In the past it was ok to write configs and 
data to the same path where the game (or application) originally 
installed. But starting with Vista or Windows 7 it's not the case 
since Program Files folder become write-protected by default.
Apr 05 2015
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 05 Apr 2015 12:39:04 +0000
schrieb "FreeSlave" <freeslave93 gmail.com>:


 On Sunday, 5 April 2015 at 11:42:42 UTC, Marco Leise wrote:

is this Windows?
 return executable_path


 
 That depends on what do you understand by data.


The data I was referring to is the stuff that is installed with
the program executable. Files that are usually the same across
installations.


 Are game's saves data too? Or content downloaded while playing
 (server-specific assets or new levels).
 In the past it was ok to write configs and 
 data to the same path where the game (or application) originally 
 installed. But starting with Vista or Windows 7 it's not the case 
 since Program Files folder become write-protected by default.


I believe modern desktops offer enough granularity to cover
each of those. For example if I was playing a game on Linux
files would go here:

/usr/share/[games/] - read-only data files.
~/.cache/ - downloaded archives, precompiled scripts, browser
            caches and other files that can be recreated or
            fetched again if they were to be deleted
~/.config/ - the user's personal configuration; may be
             overriding something in a system directory if
             desired
~/.local/share/ - pretty much a catch all for save games,
                  user created content that goes beyond the
                  scope of config files, highscores,
                  highlighting schemes in an IDE, Steam, ...

Or the other way around:

data               => /usr/share
save games         => ~/.local/share
downloaded content => ~/.local/share (or ~/.cache)
configs            => ~/.config

Windows has the Local and Roaming directories, which
serve similar but different purposes. E.g. anything machine
specific or big must not be in Roaming.
So if I generally asked for the config dir, I'd probably expect
AppData/Local on Windows and ~/.config on Linux (because I
might write a configuration that only works for this machine.)
Roaming is interesting for users that have their profiles on
servers and might switch to another workstation. So if some
configuration is "portable" and you want to create something
really fine grained you could offer that directory as an
alternative "roaming config dir" (returning null or
"~/.config" on Linux).
In any case there will be multiple results for some
directories (/usr/share, /usr/local/share) and also
several standard paths mapping to the same physical directory
(user data and user cache both map to AppData/Local on
Windows). The user needs to be made aware of this so (s)he
doesn't overwrite files in one standard path with files in a
supposedly different one.

So much for my thoughts on standard paths extreme edition. ;)

-- 
Marco
Apr 06 2015
next sibling parent reply "Kagamin" <spam here.lot> writes:
On Monday, 6 April 2015 at 21:40:28 UTC, Marco Leise wrote:

 So if some configuration is "portable" and you want to create 
 something
 really fine grained you could offer that directory as an
 alternative "roaming config dir" (returning null or
 "~/.config" on Linux).


Though it's unusual for an average programmer to figure out the 
correct usage of roaming profiles :)
Apr 07 2015
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 07 Apr 2015 11:58:58 +0000
schrieb "Kagamin" <spam here.lot>:


 On Monday, 6 April 2015 at 21:40:28 UTC, Marco Leise wrote:

 So if some configuration is "portable" and you want to create 
 something
 really fine grained you could offer that directory as an
 alternative "roaming config dir" (returning null or
 "~/.config" on Linux).


 
 Though it's unusual for an average programmer to figure out the 
 correct usage of roaming profiles :)


All they need to know is that Roaming must only contain small
and machine independent files.

(Why? Because in environments where users profiles are stored
on servers, this is the part that gets synced with those
servers. So e.g. a browser cache should not end up there, nor
should configuration that makes sense only on the current
workstation.)

Frankly, I didn't know until FreeSlave's library brought it
up. :D
Anyways my idea of portable libraries is that they should not
implement the least common denominator, but the full width of
what is available or not across different systems and offer a
common API for that, so you have the option to write well
behaving applications while rewriting as little code as
possible.

Two examples:
There is an "application menu" in OS X and in
Gnome 3, but other desktops don't provide it. I would support
it, so applications integrate nicely with these desktops and
turn it into another main menu item or taskbar right-click
menu on other desktops.
POSIX doesn't close file handles for spawned child-processes
by default, which is now considered a bad default by most.
I think a portable library should try to provide the same
functionality across systems and open files in "close on exec"
mode by default on POSIX so it is similar to Windows.

-- 
Marco
Apr 08 2015
prev sibling parent reply "FreeSlave" <freeslave93 gmail.com> writes:
On Monday, 6 April 2015 at 21:40:28 UTC, Marco Leise wrote:

 I believe modern desktops offer enough granularity to cover
 each of those. For example if I was playing a game on Linux
 files would go here:

 /usr/share/[games/] - read-only data files.
 ~/.cache/ - downloaded archives, precompiled scripts, browser
             caches and other files that can be recreated or
             fetched again if they were to be deleted
 ~/.config/ - the user's personal configuration; may be
              overriding something in a system directory if
              desired
 ~/.local/share/ - pretty much a catch all for save games,
                   user created content that goes beyond the
                   scope of config files, highscores,
                   highlighting schemes in an IDE, Steam, ...

 Or the other way around:

 data               => /usr/share
 save games         => ~/.local/share
 downloaded content => ~/.local/share (or ~/.cache)
 configs            => ~/.config

 Windows has the Local and Roaming directories, which
 serve similar but different purposes. E.g. anything machine
 specific or big must not be in Roaming.
 So if I generally asked for the config dir, I'd probably expect
 AppData/Local on Windows and ~/.config on Linux (because I
 might write a configuration that only works for this machine.)
 Roaming is interesting for users that have their profiles on
 servers and might switch to another workstation. So if some
 configuration is "portable" and you want to create something
 really fine grained you could offer that directory as an
 alternative "roaming config dir" (returning null or
 "~/.config" on Linux).
 In any case there will be multiple results for some
 directories (/usr/share, /usr/local/share) and also
 several standard paths mapping to the same physical directory
 (user data and user cache both map to AppData/Local on
 Windows). The user needs to be made aware of this so (s)he
 doesn't overwrite files in one standard path with files in a
 supposedly different one.

 So much for my thoughts on standard paths extreme edition. ;)


Just found out, there's also special Saved Games directory on 
Windows. But only starting with Vista, therefore it's not CSIDL, 
but KNOWNFOLDERID (same as for Downloads). Still I did not find 
fairly new winapi headers for D. Seems like the most use 
translation of MinGW headers which stuck at, likely, Windows XP 
era. Although we could just read some paths from registry 
avoiding calling SHGetKnownFolderPath, it's way too hacky.

Currently the only Roaming directories returned by standardpaths 
library on Windows are Templates and Applications, which is fine. 
The whole roaming thing is specific to Windows, but it would be 
useful to add separate function probably.

The same directory for configs, data and cache locations is the 
problem. I should mention it in documentation. Though for cache 
directory it returns AppData/Local/cache the same way as Qt does.
Apr 07 2015
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 07 Apr 2015 17:27:34 +0000
schrieb "FreeSlave" <freeslave93 gmail.com>:


 On Monday, 6 April 2015 at 21:40:28 UTC, Marco Leise wrote:

 I believe modern desktops offer enough granularity to cover
 each of those. For example if I was playing a game on Linux
 files would go here:

 /usr/share/[games/] - read-only data files.
 ~/.cache/ - downloaded archives, precompiled scripts, browser
             caches and other files that can be recreated or
             fetched again if they were to be deleted
 ~/.config/ - the user's personal configuration; may be
              overriding something in a system directory if
              desired
 ~/.local/share/ - pretty much a catch all for save games,
                   user created content that goes beyond the
                   scope of config files, highscores,
                   highlighting schemes in an IDE, Steam, ...

 Or the other way around:

 data               => /usr/share
 save games         => ~/.local/share
 downloaded content => ~/.local/share (or ~/.cache)
 configs            => ~/.config

 Windows has the Local and Roaming directories, which
 serve similar but different purposes. E.g. anything machine
 specific or big must not be in Roaming.
 So if I generally asked for the config dir, I'd probably expect
 AppData/Local on Windows and ~/.config on Linux (because I
 might write a configuration that only works for this machine.)
 Roaming is interesting for users that have their profiles on
 servers and might switch to another workstation. So if some
 configuration is "portable" and you want to create something
 really fine grained you could offer that directory as an
 alternative "roaming config dir" (returning null or
 "~/.config" on Linux).
 In any case there will be multiple results for some
 directories (/usr/share, /usr/local/share) and also
 several standard paths mapping to the same physical directory
 (user data and user cache both map to AppData/Local on
 Windows). The user needs to be made aware of this so (s)he
 doesn't overwrite files in one standard path with files in a
 supposedly different one.

 So much for my thoughts on standard paths extreme edition. ;)


 
 Just found out, there's also special Saved Games directory on 
 Windows. But only starting with Vista, therefore it's not CSIDL, 
 but KNOWNFOLDERID (same as for Downloads). Still I did not find 
 fairly new winapi headers for D. Seems like the most use 
 translation of MinGW headers which stuck at, likely, Windows XP 
 era. Although we could just read some paths from registry 
 avoiding calling SHGetKnownFolderPath, it's way too hacky.


The druntime Windows headers are written on a "as needed"
basis. I don't think anyone would object if you added
something you need, except maybe if it stops druntime from
working in WinXP in general. Don't know about enums that
are only meaningful on Vista and later...


 Currently the only Roaming directories returned by standardpaths 
 library on Windows are Templates and Applications, which is fine. 
 The whole roaming thing is specific to Windows, but it would be 
 useful to add separate function probably.


What do you think about OS agnostic code?

  string configPath =
    standardPath(StandardPath.config, appName);
  string roamingConfigPath =
    standardPath(StandardPath.roamingConfig, appName);

If this worked on Linux it would return the same path for both
calls, but you would avoid introducing version(Windows) switches
in user code.
The appName is there for the common use-case to get the path in
the context of the calling application. If I write a program an
want to open some "images/splash.png" in the installation
directory, I'd expect just the executable directory on Windows,
but "/usr/share/<app name>/" (or one of the overrides) on Linux.
Then again, maybe opening files is outside the scope of your
library. I could imagine it would become a full file system
abstraction layer just to support correct file lookup with all
these /usr/share, /usr/local/share, ~/.local/share and manual
overrides. :p


 The same directory for configs, data and cache locations is the 
 problem. I should mention it in documentation. Though for cache 
 directory it returns AppData/Local/cache the same way as Qt does.


-- 
Marco
Apr 08 2015
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Sunday, 5 April 2015 at 09:08:14 UTC, FreeSlave wrote:

 And github repo https://github.com/MyLittleRobo/standardpaths


If I understand meaning of PublicShare correctly, it's 
CSIDL_COMMON_APPDATA on Windows.
Apr 05 2015
parent "FreeSlave" <freeslave93 gmail.com> writes:
On Sunday, 5 April 2015 at 12:35:46 UTC, Kagamin wrote:

 On Sunday, 5 April 2015 at 09:08:14 UTC, FreeSlave wrote:

 And github repo https://github.com/MyLittleRobo/standardpaths


 If I understand meaning of PublicShare correctly, it's 
 CSIDL_COMMON_APPDATA on Windows.


Although I did not find if any spec mentions the purpose of 
Public folder on freedesktop, I think it should be used for data 
sharing in the local network.

Common AppData on Windows serves different purpose: data 
available for all users on the same computer.
Apr 05 2015
prev sibling next sibling parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 05/04/2015 11:08, FreeSlave a écrit :

 I wrote small library for getting standard paths (like Pictures, Music)
 Here's dub package http://code.dlang.org/packages/standardpaths
 And github repo https://github.com/MyLittleRobo/standardpaths

 You can see open issues on github. Please, participate in discussions if
 you're interested. The biggest problem now is OS X support
 https://github.com/MyLittleRobo/standardpaths/issues/4

 You can generate documentation with dub build --build=docs.


Nice,

IMO it can have his place in phobos, cause at least all GUI applications 
have to retrieve one or more of those paths.

This kind of API are important to ease fast prototyping applications or 
scripts.
Apr 06 2015
prev sibling next sibling parent reply "FreeSlave" <freeslave93 gmail.com> writes:
OS X support added. It dynamically loads some Carbon functions. 
But path to the framework is hardcoded as 
"/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore"
I'm not sure that it's ok, need advice from OS X programmers.
May 08 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-05-08 22:37, FreeSlave wrote:

 OS X support added. It dynamically loads some Carbon functions. But path
 to the framework is hardcoded as
 "/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore"

 I'm not sure that it's ok, need advice from OS X programmers.


The documentation [1] recommends using "URLsForDirectory:inDomains:", 
but that is an Objective-C method.

[1] 
https://developer.apple.com/library/ios/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/AccessingFilesandDirectories/AccessingFilesandDirectories.html#//apple_ref/doc/uid/TP40010672-CH3-SW1

-- 
/Jacob Carlborg
May 10 2015
prev sibling parent "Baz" <bb.temp gmx.com> writes:
On Sunday, 5 April 2015 at 09:08:14 UTC, FreeSlave wrote:

 I wrote small library for getting standard paths (like 
 Pictures, Music)
 Here's dub package http://code.dlang.org/packages/standardpaths
 And github repo https://github.com/MyLittleRobo/standardpaths

 You can see open issues on github. Please, participate in 
 discussions if you're interested. The biggest problem now is OS 
 X support https://github.com/MyLittleRobo/standardpaths/issues/4

 You can generate documentation with dub build --build=docs.


It's a small but useful lib. Added to metad for Coedit:
https://github.com/BBasile/metad/blob/master/.gitmodules#L82
May 08 2015