www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - C tips (again)

reply bearophile <bearophileHUGS lycos.com> writes:
In the past I have probably already discussed about this link here:
http://users.bestweb.net/~ctips/

That site contains tips as precious as gold, even for non-C programmers.

You can easely see that those C tips (ctips) are written by a person that has a
very large experience in programming C, more than most other person I have read
about or met.

I have never appreciated the famous book by Kernighan&Ritchie about C (despite
I use today still, mostly as a reference for the C std lib). But I have
appreciated those ctips (that are almost a book) as soon I have seen that site.

So far about C I haven't found a book or site better than those ctips, if you
know one of such references please tell me.

Those ctips are nice because they show many creative ideas (it shows things I
have believed as nearly impossible/very hard to do with D).
I like those ctips also because it shows idioms that are at the same time:
- Low level (so they are usually very efficient, both in speed and final
program size. And speed is one of the things that give power to a programmer);
- Designed to be used to create big systems, like to create C programs 100_000
lines long or more (so those idioms are designed to manage/kill high
complexity);
- Designed to be write reliable software (so they are designed to avoid bugs.
The C language is a very bug-prone language, but those ctips show ways to avoid
or quickly hunt down most bugs).

Putting together speed (a kind of power) with anti-bug/anti-complexity makes
those idioms almost universal for computers, so probably some of the ideas they
rest on can be used in other low level languages beside C.

A problem is that even after reading those ctips two times, I am partially
unable to keep them in memory and to use them. I have even troubles in copying
them down from the site pages, because there are too many details to keep in
memory, and because they are designed with an extreme rigorousness, like by an
old Tibetan monk (on the other hand I am sure that if you want to write a 200K
lines long C program, and you want a certain probability to see it actually
run, you must use more rigor than I have ever used in my small programs). The
author of those ctips can keep such idioms in memory because he has a ton of
practice of C programming.

From a certain point of view you can think of a medium/high level as
D/Python/Ruby/Java as a collection of idioms of low level languages, that now
you can use at high level (OOP can be seen as a collection of idioms in C. In
old structured programming can be seen as an idiom. In the beginning even
routines and functions were seen as idioms in assembly programming. There was
even people that have discussed about the pro/cons of using routines!). That
said, to design a good medium/high language (I consider D a medium level one)
you must choose the right idioms, the best ones.

Those ctips are a collection of good low-level idioms. I think may be a good
starting point to design a medium-level language (with power similar to C). So
they can be useful to design part of D too. Currently D has already added some
of those idioms.

--------------------

The following is a small example of one of those idioms, that I have used in a
C program to have dynamic strings. I have seen that it's faster than
alternative solutions like having a struct with size/capacity/pointer that
points to a block of memory.

I have seen this idiom works in D too (and Mystring.sizeof == 8 still) but you
can't use this idiom in D, you must always use -release.

This code is written by me, so it's NOT representative of the kind of code you
can find in those ctips, my code is surely much worse.

This stuff is error-prone, and the author of ctips agrees with this. He never
suggests to use things like this in this way. He suggests to add lot of other
stuff here, to make this thing safer, sentinels to see if there are overflows,
reflection means, more type flexibility, etc. You can't use ctips idioms half
way, you have to use them fully, and this is NOT easy.

int max2(int a, int b) {
    return a > b ? a : b;
}

typedef struct { // sizeof(Mystring) == 8
    int size;
    int capacity;
    char data[0];
} Mystring;

Mystring* mystring_new(int capacity) {
    int new_capacity = max2((int)sizeof(int), capacity);
    int total_mem = sizeof(Mystring) + sizeof(char) * new_capacity;

    Mystring* str = malloc(total_mem);
    if (str == NULL) {
        printf("Error allocating %d bytes.\n", total_mem);
        exit(-1);
    }

    str->size = 0;
    str->capacity = new_capacity;
    return str;
}

Mystring* mystring_append_char(Mystring* str, char c) {
    assert(str != NULL && str->capacity > 0);
    if (str->capacity <= str->size) {
        // not enough space
        str->capacity *= 2;
        int new_total_mem = sizeof(Mystring) + str->capacity * sizeof(char);
        str = realloc(str, new_total_mem);
        if (str == NULL) {
            printf("Error allocating %d bytes.\n", new_total_mem);
            exit(-1);
        }
    }
    str->data[str->size] = c;
    str->size += 1;
    return str;
}


If you follow those ctips fully, you end having more reflection in such data
structures than usually D gives you. That's why I have said it may be good to
have many of those idioms as built-ins in a medium-level language.

Bye,
bearophile
May 01 2009
next sibling parent reply superdan <super dan.org> writes:
bearophile Wrote:

 In the past I have probably already discussed about this link here:
 http://users.bestweb.net/~ctips/
 
 That site contains tips as precious as gold, even for non-C programmers.

d00d. u first messed with k&r which are Gods. any1 mess'n' with Gods better bring around some fuckin' supergod instead. the trailin' empty array trick is so ol' it smells like ol' people. been thru first 19 pieces of advice. nuttin' new or remotely interestin'. wut i saw was some lame shit that puts a grown man 2 sleep. use assert, macros, & checks. wtf. then got interrupted. the 70s called. they wanted they lame c idioms back.
May 01 2009
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
superdan:
 d00d. u first messed with k&r which are Gods. any1 mess'n' with Gods better
bring around some fuckin' supergod instead.<

I have not appreciated that book, I don't care of what you say. I am generally able to see good books when I see then. I guess that author has more programming experience than all my teachers combined.
the trailin' empty array trick is so ol' it smells like ol' people.<

Of course few or none of those things are new. C is around for a lot of time. The author never says he's inventing things. But the way he is combining those things, the rigor he shows, creates something I have not seen elsewhere. Note that he also shows the two-empty array trick (not found elsewhere by me) and then adds sentinels, reflection, type genericity, several tricks to memory safety, etc. In most of the good C code I see around there's no even 1/5 of those qualities (that become qualities almost only in bigger programs). This is an example of very good C code: http://svn.python.org/view/*checkout*/python/trunk/Modules/_collectionsmodule.c?revision=70298&content-type=text%2Fplain It's better than most C code you see around, yet it's far from the code shown in the ctips.
been thru first 19 pieces of advice. nuttin' new or remotely interestin'.<

I don't agree. I have some experience of C programming, and yet, I have found several interesting things.
wut i saw was some lame shit that puts a grown man 2 sleep.<

I guess Ruby coding is more sexy. This is sometimes more boring that Ada coding, but you can't use Ruby for some of those things.
use assert, macros, & checks. wtf. then got interrupted. the 70s called. they
wanted they lame c idioms back<

Among those idioms there are some things, like certain reflection, that D isn't doing now. If you write to write/fix big programs in C today you need those things. And more modern languages like C# are a collection of some of those idioms, as I have explained. Note that I am planning in creating a new module for Python 2.7, it has to be written in C, so I need C today too, it seems. So I agree with nearly nothing you have (badly) expressed. Bye, bearophile
May 01 2009
next sibling parent reply superdan <super dan.org> writes:
bearophile Wrote:

 superdan:
 d00d. u first messed with k&r which are Gods. any1 mess'n' with Gods better
bring around some fuckin' supergod instead.<

I have not appreciated that book, I don't care of what you say.

fine. personal preference is kewl.
 I am generally able to see good books when I see then.

where the mother fuck did that come from. first u say preference is relative. /u/ didn't like k&r. fine. then u make it fuckin' absolute. if k&r was any good, u bet ur ass bearophile had figured that out. that's fucked up like a hooker in downtown bangkok.
 I guess that author has more programming experience than all my teachers
combined.

experience is nuttin'. it's what u learn from it. half the dimwits in my office have more of it than me. & they dun miss any opportunity to fuckin' /remind/ me whenever their incompetence comes forth. `superdan u should know i have 18 years of experience designing shit'. `yeah then why is ur design fucked up with a dry cock up its ass. kiss my black ass if in 18 years u didn't learn any.' shit. if i hear one more dood invokin' experience as central argument i swear i go fuckin' postal.
the trailin' empty array trick is so ol' it smells like ol' people.<

Of course few or none of those things are new. C is around for a lot of time. The author never says he's inventing things. But the way he is combining those things, the rigor he shows, creates something I have not seen elsewhere.

u havent been around much, that's the prob. check this shit out. http://www.amazon.com/Hackers-Delight-Henry-S-Warren/dp/0201914654. then come back.
May 01 2009
parent bearophile <bearophileHUGS lycos.com> writes:
superdan:
experience is nuttin'. it's what u learn from it.<

This is generally right, exercising yourself a lot isn't enough, because you have to do it in a smart way, for example trying things a bit more difficult than the things you already know, etc. But experience can also be positive: regarding the ctips the experience of author allows him to follow rigorous and complex idioms in a very tidy way.
u havent been around much, that's the prob.<

I know the February 2002 edition of Hacker's Delight. It mostly explains numerical/bitwise things, like: - Power-of-2 boundaries and bounds checking - Rearranging bits and bytes - Integer division and division by constants - Some elementary functions on integers - Gray code - Hilbert's space-filling curve More or less none of those things are present in the ctips. Bye, bearophile
May 01 2009
prev sibling next sibling parent reply Steve Teale <steve.teale britseyeview.com> writes:
bearophile Wrote:
 
 So I agree with nearly nothing you have (badly) expressed.
 
 Bye,
 bearophile

Who is this guy?" He must be putting his messages through one of those web sites that translate into strange dialects - fry my ass! I'm with you, reading K&R back in the 80s was a pivotal point my life!
May 01 2009
next sibling parent reply superdan <super dan.org> writes:
Steve Teale Wrote:

 bearophile Wrote:
  
 So I agree with nearly nothing you have (badly) expressed.
 
 Bye,
 bearophile

Who is this guy?" He must be putting his messages through one of those web sites that translate into strange dialects - fry my ass!

fer tat i recommend da george foreman grill. i'm a bad motherfucker. fer the most part u can just ignore me.
 I'm with you, reading K&R back in the 80s was a pivotal point my life!

u haven't really read his post have ya. to ur chagrin u ain't with da nicey guy, u r with the bad motherfucker. sorry to break them bad newz.
May 01 2009
parent Christopher Wright <dhasenan gmail.com> writes:
superdan wrote:
 Steve Teale Wrote:
 
 bearophile Wrote:
  
 So I agree with nearly nothing you have (badly) expressed.

 Bye,
 bearophile


fer tat i recommend da george foreman grill.

Yes, but will it fritter my wig?
May 02 2009
prev sibling parent downs <default_357-line yahoo.de> writes:
Steve Teale wrote:
 bearophile Wrote:
  
 So I agree with nearly nothing you have (badly) expressed.

 Bye,
 bearophile

Who is this guy?" He must be putting his messages through one of those web sites that translate into strange dialects - fry my ass! I'm with you, reading K&R back in the 80s was a pivotal point my life!

May 01 2009
prev sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
bearophile wrote:
 Among those idioms there are some things, like certain reflection,
 that D isn't doing now.

C has reflection?
 Note that I
 am planning in creating a new module for Python 2.7, it has to be
 written in C, so I need C today too, it seems.

Why does it have to be written in C? D's C interface works both ways.
May 01 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright Wrote:
C has reflection?<

Yes, of course, if you implement it. C has OOP too if you implement all that machinery. C is quite efficient, so it gives you the power to implement almost everything (the problem is, often things get too much complex if you want to implement them manually in C, so sometimes you are unable to use such power. That's the purpose of the idioms that are bricks of medium/high level languages).
Why does it have to be written in C?<

CPython is written in C (and a bit of Python for some libs). I'd like to add a module to the Pythonh 2.7 standard lib, and to be efficient it can't be written in Python. So I have to use C. D isn't allowed in the CPython codebase. Bye, bearophile
May 01 2009
parent reply Walter Bright <newshound1 digitalmars.com> writes:
bearophile wrote:
 Walter Bright Wrote:
 C has reflection?<

Yes, of course, if you implement it. C has OOP too if you implement all that machinery. C is quite efficient, so it gives you the power to implement almost everything

Why couldn't you do the exact same thing in D?
 Why does it have to be written in C?<

CPython is written in C (and a bit of Python for some libs). I'd like to add a module to the Pythonh 2.7 standard lib, and to be efficient it can't be written in Python. So I have to use C. D isn't allowed in the CPython codebase.

So it's a political rather than technical requirement. That I can understand <g>.
May 01 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:
Why couldn't you do the exact same thing in D?<

Those ctips show some tricks done with the preprocessor that I don't know yet how to do with D1, and there are some situations where in D I'd like to be able to locally disable array bound checks (this is doable in Delphi/FreePascal), like in the data array here: typedef struct { int size; int capacity; char data[0]; } Mystring; But D is more or less a superset of C, so a D programmer may be able to implement all those things, as you say. The main point of my original post is that some of those ctips may be added as built-in idioms to D. Regarding possible things to be added to D: recently I have heard that some people is thinking about creating a small C compiler that supports some nonstadard low-level features, to be used to compile the future Linux kernel: http://lkml.org/lkml/2009/4/22/78 If such project goes on, then it may show other things that may be useful in D if D wants to be fit to write kernels too. Bye, bearophile
May 01 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
bearophile wrote:
 Walter Bright:
 Why couldn't you do the exact same thing in D?<

Those ctips show some tricks done with the preprocessor that I don't know yet how to do with D1, and there are some situations where in D I'd like to be able to locally disable array bound checks (this is doable in Delphi/FreePascal), like in the data array here: typedef struct { int size; int capacity; char data[0]; } Mystring;

(untested) struct Mystring { size_t size, capacity; char* pdata() { return cast(char*) (&this + 1); } char[] data() { return pdata()[0 .. size]; } }
 But D is more or less a superset of C, so a D programmer may be able to
implement all those things, as you say.
 The main point of my original post is that some of those ctips may be added as
built-in idioms to D.

I've had a nice "blast from the past" feeling while skimming the document, but really it's more of a proof that with ingenuity and discipline a bear can be made to dance. I don't see how that stuff could help D any more than a coconut and two straws could help a radar receiver. I have been mightily disappointed (essentially my respect for the author has crashed into the ground) when I saw that the entire optimization example, although clearly a lot of time and effort has been put into it, had absolutely no numbers whatsoever.
 Regarding possible things to be added to D: recently I have heard that some
people is thinking about creating a small C compiler that supports some
nonstadard low-level features, to be used to compile the future Linux kernel:
 http://lkml.org/lkml/2009/4/22/78
 If such project goes on, then it may show other things that may be useful in D
if D wants to be fit to write kernels too.

IMHO all D needs for writing kernel-grade code is a way to code cleanly without the GC. Andrei
May 01 2009
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Denis Koroskin:
 Mystring s = ...;
 char c = s.data.ptr[42];

Thank you for your nice suggestion. In Python newsgroups people usually before suggesting any solution try to run it to see if it works correctly (this isn't always possible). The following program doesn't work to me when WITH_PTR is true (Error: Access Violation) (to be compiled with -release when WITH_PTR is false): import std.c.stdio: printf; import std.c.stdlib: malloc, exit; const bool WITH_PTR = 1; struct Mystring { int size; int capacity; char data[0]; } void main() { string s = "hello, how are you?"; int total_len = Mystring.sizeof + s.length + 1; auto str = cast(Mystring*)malloc(total_len); if (str is null) exit(-1); static if (WITH_PTR) { foreach (i, c; s) str.data.ptr[i] = c; str.data.ptr[s.length] = '\0'; printf("%s\n", str.data.ptr); } else { foreach (i, c; s) str.data[i] = c; str.data[s.length] = '\0'; printf("%s\n", &(str.data)); } } Bye, bearophile
May 03 2009
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 01 May 2009 11:36:08 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:

 superdan:
 d00d. u first messed with k&r which are Gods. any1 mess'n' with Gods  
 better bring around some fuckin' supergod instead.<

I have not appreciated that book, I don't care of what you say. I am generally able to see good books when I see then. I guess that author has more programming experience than all my teachers combined.

Yeah, he only, you know, *invented* C ;) Despite this, I pretty much do as you say, only use the K&R book for reference. And it works out just fine. It always reminds me of how simple C is, that it fits in such a small book (standard library and all). I almost never have to look at online docs. -Steve
May 01 2009
parent superdan <super dan.org> writes:
Steven Schveighoffer Wrote:

 On Fri, 01 May 2009 11:36:08 -0400, bearophile <bearophileHUGS lycos.com>  
 wrote:
 
 superdan:
 d00d. u first messed with k&r which are Gods. any1 mess'n' with Gods  
 better bring around some fuckin' supergod instead.<

I have not appreciated that book, I don't care of what you say. I am generally able to see good books when I see then. I guess that author has more programming experience than all my teachers combined.

Yeah, he only, you know, *invented* C ;)

he was referrin' 2 the famous nobody who wrote them tips he sleeps under his pillow with.
May 01 2009
prev sibling next sibling parent Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Fri, May 1, 2009 at 5:37 PM, Walter Bright
<newshound1 digitalmars.com> wrote:

 Note that I
 am planning in creating a new module for Python 2.7, it has to be
 written in C, so I need C today too, it seems.

Why does it have to be written in C? D's C interface works both ways.

Yeah.. heard of Pyd? It's made expressly for making Python extension modules.
May 01 2009
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 02 May 2009 03:56:05 +0400, bearophile <bearophileHUGS lycos.com> wrote:

 Walter Bright:
 Why couldn't you do the exact same thing in D?<

Those ctips show some tricks done with the preprocessor that I don't know yet how to do with D1, and there are some situations where in D I'd like to be able to locally disable array bound checks (this is doable in Delphi/FreePascal), like in the data array here: typedef struct { int size; int capacity; char data[0]; } Mystring;

Mystring s = ...; char c = s.data.ptr[42];
May 02 2009
prev sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Sun, 03 May 2009 14:19:08 +0400, bearophile <bearophileHUGS lycos.com> wrote:

 Denis Koroskin:
 Mystring s = ...;
 char c = s.data.ptr[42];

Thank you for your nice suggestion. In Python newsgroups people usually before suggesting any solution try to run it to see if it works correctly (this isn't always possible). The following program doesn't work to me when WITH_PTR is true (Error: Access Violation) (to be compiled with -release when WITH_PTR is false): import std.c.stdio: printf; import std.c.stdlib: malloc, exit; const bool WITH_PTR = 1; struct Mystring { int size; int capacity; char data[0]; } void main() { string s = "hello, how are you?"; int total_len = Mystring.sizeof + s.length + 1; auto str = cast(Mystring*)malloc(total_len); if (str is null) exit(-1); static if (WITH_PTR) { foreach (i, c; s) str.data.ptr[i] = c; str.data.ptr[s.length] = '\0'; printf("%s\n", str.data.ptr); } else { foreach (i, c; s) str.data[i] = c; str.data[s.length] = '\0'; printf("%s\n", &(str.data)); } } Bye, bearophile

I use that trick quite often and it works. The problem here is, DMD initialize 0-length static arrays' ptr with null: void main() { auto str = cast(Mystring*)malloc(42); assert(str.data.ptr !is null); // fails } I believe this issue needs to be posted to bugzilla.
May 03 2009
prev sibling parent Georg Wrede <georg.wrede iki.fi> writes:
bearophile wrote:
 From a certain point of view you can think of a medium/high level as
 D/Python/Ruby/Java as a collection of idioms of low level languages,
 that now you can use at high level (OOP can be seen as a collection of
 idioms in C. In old structured programming can be seen as an idiom. In
 the beginning even routines and functions were seen as idioms in
 assembly programming.

The same has happened in hardware. Now we have even systems-on-a-chip. All you need is solder the connectors on (for say USB, keyboard, etc.), and you have an entire computer.
 There was even people that have discussed about
 the pro/cons of using routines!).

Yeah, always naysayers. OTOH it's good, since then the merits of the new thinking get studied better.
 That said, to design a good medium/high language (I consider D a
 medium level one) you must choose the right idioms, the best ones.

I think D is aiming to be a /both/ low, medium, and high level language. And I think that's the right thing to do. Then you can use almost the same syntax all the time, while still being able to use the particular level that the current source line of your code needs. I'd call that luxury, simply because I believe D is almost unique here. Had I a medium level language, then I'd have to resort to a high and a low level language to do some of my application. That is a serious PITA, like I guess everyone agrees.
 Those ctips are a collection of good low-level idioms. I think may be
 a good starting point to design a medium-level language (with power
 similar to C). So they can be useful to design part of D too. Currently
 D has already added some of those idioms.

May 02 2009