diff --git a/book/2003-12.txt b/book/2003-12.txt new file mode 100644 index 0000000..40764d3 --- /dev/null +++ b/book/2003-12.txt @@ -0,0 +1,4345 @@ +\start +Date: Sun, 30 Nov 2003 21:30:15 -0500 +From: Tim Daly +To: Bertfried.Fauser@uni-konstanz.de +Subject: [Axiom-developer] Re: AXIOM from cvs does not compile + +> TANGLE=/home/fauser/axiom/mnt/bin/bin/lib/notangle + ^^^^^^^ +hummm... which means that SPADBIN must be bad: + +> SPADBIN=/home/fauser/axiom/mnt/bin/bin + +but this is a constructed variable ending in SYS: + +> SYS=bin + +Therefore the AXIOM variable must have been: + +AXIOM=/home/fauser/axiom/mnt/bin + +but should have been: + +AXIOM=/home/fauser/axiom/mnt/linux + +\start +Date: Mon, 1 Dec 2003 10:55:19 +0000 +From: Mike Dewar +To: axiom-developer@nongnu.org +Subject: Re: [Axiom-developer] Axiom Book (Jenks & Sutor) + +Sorry for the delay in responding to people who asked for copies, but I +was out of the office at a meeting. The winners are Juergen Weiss, +Michel Lavaud, Martin Rubey, Luca Giuzzi and Hans Peter Würmli, who +should receive copies shortly. Sorry to the unlucky applicants, in the +unlikely event we find any more then I'll let you know. + +Several people offered money for these, I should have made it clearer +that the books were free (we would just have thrown them away if a +good home wasn't available). + +Kind regards, Mike. + +On Fri, Nov 21, 2003 at 09:40:58AM +0000, Mike Dewar wrote: +> While clearing out a cupboard, one of my colleagues came across a +> handful of copies of Jenks & Sutor. If anybody on this list would like +> one then please send me your snail mail address and we'll ship it to you +> wherever you are, even if its the Netherlands :-) [1] First come, first +> served. +> +> Cheers, Mike. +> +> [1] this is a football reference. Holland trounced Scotland on +> Wednesday :-( + +\start +Date: Mon, 01 Dec 2003 22:53:16 +0100 +From: David MENTRE +To: daly@idsi.net +Subject: Re: [Axiom-developer] Questions (Bugs?) +Cc: dirnstor@web.de + +root writes: + +> Please add it as a bug so it doesn't get lost. + +Ok. I've added Stephan's third bug as #6903. +http://savannah.nongnu.org/bugs/index.php?func=detailitem&item_id=6903 + +\start +Date: Mon, 01 Dec 2003 22:32:40 +0100 +From: David MENTRE +To: daly@idsi.net +Subject: [Axiom-developer] Re: About pamphlets and literate programming: how to identify identification levels? +Cc: axiom@tenkan.org, axiom-developer@nongnu.org + +Hello Tim, + +root writes: + +> I'm currently working on a demonstration pamphlet based on the paper called +> Primes is in P. I'm hoping to finish it and present it at a workshop in +> Italy next week. It is an ideal example of working from the theory all +> the way down to the code. + +Ok. I'm not sure now that I made my point very clearly but I'll wait +until you make available this paper and I'll ask you further questions +if needed at that time. + +\start +Date: Mon, 1 Dec 2003 17:49:52 +0100 (CET) +From: Bertfried Fauser +To: Tim Daly +Subject: [Axiom-developer] Re: AXIOM from cvs does not compile +Cc: Bertfried.Fauser@uni-konstanz.de + +On Sun, 30 Nov 2003, Tim Daly wrote: + +Dear Tim, + +sorry for my stupidity, + +> Therefore the AXIOM variable must have been: +> +> AXIOM=/home/fauser/axiom/mnt/bin + +your guess was quite right, even if the AXIOM variable was different, it +was not OK. I wonder why I was able to compile AXIOM the first time.... + +All in all, a few other PATH issues seemed to fail so that AXIOMsys was +not functional. Its fixed and --heureka- this solves my TeXmacs problem +too. + +\start +Date: 01 Dec 2003 17:47:19 -0500 +From: Camm Maguire +To: Tim Daly +Subject: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Greetings! Tim, the -Wa,--execstack option is trivial to implement +and is sure to work. It would be nice to have a configure test to +determine when it is necessary, but all gcc I have available seem to +take the option. Even though it is not the desirable long term option +as the poster states, a final solution must await a fix to emacs' +unexec, at which point we can move DBEGIN to an internal variable, and +test if holes are permitted in the heap. So at this point I recommend +punting and disabling exec-shield until a work-around for unexec is at +hand. Your thoughts? + +Take care, + +Tim Daly writes: + +> Some hints from the Fedora Core mailing list about making programs +> run under exec-shield. +> +> Tim +> +> From: Ingo Molnar +> Subject: Re: Executable memory: further programs that fail +> To: Fedora Development +> Date: Mon, 24 Nov 2003 05:42:27 -0500 (EST) +> +> +> On Sat, 22 Nov 2003, Gerard Milmeister wrote: +> +> > So is it alright to include in Fedora packages that require exec-shield +> > to be turned off? Should there be a wrapper-script that calls the main +> > executable with 'setarch'? +> +> all 'third party' apps will run with exec-shield disabled by default. +> +> _Iff_ a package is recompiled under Fedora (using Fedora's gcc and +> binutils) then you might end up with an app that has exec-shield enabled +> although the app itself for whatever reason has some executability +> assumption. +> +> In this case there are myriads of ways to make the app work again: +> +> - use 'setarch i386 ' [or rather, use the 'i386 ' shortcut] +> +> - add this gcc option to the CFLAGS in the Makefile: +> +> -Wa,--execstack +> +> - add the following oneliner anywhere in the app's source in an assembly +> file to disable exec-shield: +> +> .section .note.GNU-stack, "x", @progbits; .previous +> +> - disable exec-shield globally, put "kernel.exec-shield = 0" into +> /etc/sysctl.conf. +> +> - the preferred solution: fix the app itself to not assume executability +> of non-executable regions or not hardcode any VM properties such as +> 'mmaps start at 2 GB'. Both assumptions are illegal and these apps will +> likely break on 64-bit architectures too, so they should be fixed. Note +> that typically the changes needed are quite simple. +> +> in any case, please keep reporting apps that need an executable stack, we +> want to track them all down and fix them for good. Fortunately 99.9% of +> the 5000+ binaries in a full Fedora Core 1 install work out of box. +> +> note that there are also prelink related problems, which we want to know +> about and fix just as much, but which should not be confused with +> exec-shield problems. Sometimes these do get mixed up. +> +> (in any case, please also read the extensive description provided by +> Roland McGrath's "exec-shield mmap & brk randomization" email.) +> +> Ingo + +\start +Date: Mon, 1 Dec 2003 19:21:37 -0500 +From: "Page, Bill" +To: "'Bertfried.Fauser@uni-konstanz.de'" +Subject: RE: [Axiom-developer] Re: AXIOM from cvs does not compile + +On Monday, December 01, 2003 11:50 AM Bertfried Fauser +[mailto:fauser@spock.physik.uni-konstanz.de] wrote: +> +> ... Its fixed and --heureka- this solves my TeXmacs +> problem too. + +Excellent! + +Now if only doing theoretical physics was this easy ... + + +\start +Date: Mon, 1 Dec 2003 07:51:16 -0500 +From: Tim Daly +To: Bertfried.Fauser@uni-konstanz.de +Subject: [Axiom-developer] Re: AXIOM from cvs does not compile +Cc: Bertfried.Fauser@uni-konstanz.de + +The time has come to remove the AXIOM variable. +And apply Juergen's case patch. +Unfortunately I'll be in Italy this week trying to make the case +for literate programming but I'll move these up on the todo list. +I suppose it would even be useful to publish the current todo list. +I'll put it on the todo list :-) + +\start +Date: Tue, 2 Dec 2003 08:25:43 -0500 +From: root +To: camm@enhanced.com +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Camm, + +I believe you can query the exec-shield state by + + cat /proc/sys/kernel/exec-shield + +and probably fail to build if it is set to 1. +The user will have to be root to set it which is not required +otherwise unless they want to do a make install. + +I can add it to the Axiom configure script. + +I've raised objections to exec-shield on the fedora mailing +list (I'm a developer). The code has only been added in the last +two weeks so now is the time to complain. However the complaints +have been dismissed. I've read all the docs they pointed out to +me and nowhere is it explained how it helps security. It does +hurt memory management but the basic response is "hey, nobody +REQUIRES memory to be contiguous". It's about like putting chairs +on the football field and claiming that there is nothing in the +rules that says this is illegal. Of course every team at every +game will have to play "around" the random chairs. + +I'm not sure how to fix unexec. It appears that a possible solution +will be to dynamically rebuild memory. One poster claimed that the +"randomized shared library locations" will only happen in a 100Meg +area. A possible strategy is to find the highest allocated byte +in the highest shared library, mark the area below as part of the +image and only allocate above that mark. Unexec could "save" the +shared libraries as part of the image even though they are relinked +again on restart. + +In general I'm rather confused about the whole issue. I'll have to +read the unexec code in Emacs and get a real clue. + +\start +Date: 02 Dec 2003 15:08:22 +0100 +From: Peter Simons +To: axiom-developer@nongnu.org +Subject: Re: [Axiom-developer] Re: [Re: Executable memory: further programs that fail] + +root writes: + + > I believe you can query the exec-shield state by + + > cat /proc/sys/kernel/exec-shield + +The more I read about this, the more I get the impression +that "exec-shield" is something entirely different than the +"ProPolice" system I'm running here. + +Just curious, are there any ideas how to fix the issues with +PaX and ProPolice? (Other than disabling them.) + +\start +Date: Tue, 2 Dec 2003 09:15:50 -0500 +From: root +To: simons@cryp.to +Subject: Re: [Axiom-developer] Re: [Re: Executable memory: further programs that fail] + +Peter, + +ProPolice was mentioned on the Fedora-devel-list but I don't +believe they mentioned a way to disable it. The exec-shield discussion +centered around the random shared library placement issue and, as it +was off-topic, I let it drop. + +exec-shield can be disabled with: + +echo 0 >/proc/sys/kernel/exec-shield + +\start +Date: 02 Dec 2003 09:53:43 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Hi Tim! + +root writes: + +> Camm, +> +> I believe you can query the exec-shield state by +> +> cat /proc/sys/kernel/exec-shield +> +> and probably fail to build if it is set to 1. +> The user will have to be root to set it which is not required +> otherwise unless they want to do a make install. +> + +OK, I'll probe this file, query its value, and add -Wa,--execstack to +the flags for now. Will let you know when this is done. + +> I can add it to the Axiom configure script. +> +> I've raised objections to exec-shield on the fedora mailing +> list (I'm a developer). The code has only been added in the last + +Thanks for your lisp defense efforts! :-) + +> two weeks so now is the time to complain. However the complaints +> have been dismissed. I've read all the docs they pointed out to +> me and nowhere is it explained how it helps security. It does +> hurt memory management but the basic response is "hey, nobody +> REQUIRES memory to be contiguous". It's about like putting chairs +> on the football field and claiming that there is nothing in the +> rules that says this is illegal. Of course every team at every +> game will have to play "around" the random chairs. + +>From our conversation with Roland, there appears to be one area they +would consider changing, and this would help us a lot -- to forgo brk +randomization, as apart from mmap, malloc, and ld.so randomization. +Roland thought this randomization was a violation of the 'spec' of brk +(I agree) and therefore should not be implemented, rather people +should be encouraged to not use brk in favor of more 'modern' memory +allocation routines. If I were in your position of influence, I would +lobby for this, and not waste your wind trying to blow down +exec-shield in its entirety. This simple change would solve virtually +all our problems. It would however leave a non-randomized allocation +path on the system, which may not be desirable from the point of view +of security. Just to be clear, we might be able to work around a +randomized brk as long as someone fixes unexec, but it will leave +holes in the heap and diminish the available memory to the user. + +There are security gains to be had from this approach in general, and +that is that an attacker cannot rely on a buffer overflow exploit +being in a certain position in memory. I've never studied the +construction of such exploits, but it would seem this would pose an +insuperable obstacle. + +> +> I'm not sure how to fix unexec. It appears that a possible solution +> will be to dynamically rebuild memory. One poster claimed that the +> "randomized shared library locations" will only happen in a 100Meg +> area. A possible strategy is to find the highest allocated byte +> in the highest shared library, mark the area below as part of the +> image and only allocate above that mark. Unexec could "save" the +> shared libraries as part of the image even though they are relinked +> again on restart. +> + +Luckily emacs seems to have sufficient importance that someone else +will likely do this for us. I've been treating unexec as a black box +myself. + +Take care, + +> In general I'm rather confused about the whole issue. I'll have to +> read the unexec code in Emacs and get a real clue. + +\start +Date: 02 Dec 2003 10:22:29 -0500 +From: Camm Maguire +To: Tim Daly +Subject: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Greetings! OK, the -Wa,--execstack changes are in cvs, both HEAD and +2.6.1 as I figure you might want a stable series gcl on fedora. These +options can be added explicitly whether or not the proc file exists +and contains non-zero by setting the CFLAGS environment variable when +running configure, i.e + +CFLAGS="-Wa,--execstack" ./configure .... + +This will work on your existing version provided you don't specify +--enable-debug. It will work even with this option in the configure +version just checked into cvs. + +\start +Date: 02 Dec 2003 10:26:11 -0500 +From: Camm Maguire +To: Peter Simons +Subject: Re: [Gcl-devel] Re: [Axiom-developer] Re: [Re: Executable memory: further programs that fail] + +Greetings! + +Peter Simons writes: + +> root writes: +> +> > I believe you can query the exec-shield state by +> +> > cat /proc/sys/kernel/exec-shield +> +> The more I read about this, the more I get the impression +> that "exec-shield" is something entirely different than the +> "ProPolice" system I'm running here. +> +> Just curious, are there any ideas how to fix the issues with +> PaX and ProPolice? (Other than disabling them.) +> + +While I do not have time right now to search for a solution myself, +if you can discover a simple one and explain it clearly to me, I'll +try to get it into GCL. + +\start +Date: Tue, 2 Dec 2003 10:54:00 -0500 +From: root +To: camm@enhanced.com +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Please remind me again of the correct cvs flags to check out the 2.6.1 latest. + +\start +Date: Tue, 2 Dec 2003 10:53:09 -0500 +From: root +To: camm@enhanced.com +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +>From our conversation with Roland, there appears to be one area they +>would consider changing, and this would help us a lot -- to forgo brk +>randomization, as apart from mmap, malloc, and ld.so randomization. +>Roland thought this randomization was a violation of the 'spec' of brk +>(I agree) and therefore should not be implemented, rather people +>should be encouraged to not use brk in favor of more 'modern' memory +>allocation routines. If I were in your position of influence, I would +>lobby for this, and not waste your wind trying to blow down +>exec-shield in its entirety. This simple change would solve virtually +>all our problems. It would however leave a non-randomized allocation +>path on the system, which may not be desirable from the point of view +>of security. Just to be clear, we might be able to work around a +>randomized brk as long as someone fixes unexec, but it will leave +>holes in the heap and diminish the available memory to the user. + +I've only been attacking the brk randomization issue, not all of +exec-shield. The claim is that brk randomization will "only" allocate +within 100M of memory. (We elders-of-the-field consider 1M to be a +significant amount of memory :-) ) Roland appears to be unmoved by +the discussion so I suppose that we may have to fall back on static +linking as the only acceptable solution. + +I forget why static linking failed but I'll have to get up to speed +on that again. I'll be at a workshop in italy until next tuesday so +nothing is going to happen until then. When I get back I'll figure +out how to statically link Axiom so the issue goes away. + +>There are security gains to be had from this approach in general, and +>that is that an attacker cannot rely on a buffer overflow exploit +>being in a certain position in memory. I've never studied the +>construction of such exploits, but it would seem this would pose an +>insuperable obstacle. + +The basic idea is that you find an exploit in library code, find out +where the dynamic library gets loaded, put a bogus "return address" +onto the stack which branches to the library code and run the exploit. +Since you don't know where the dynamic library will get loaded you +can't hard-code a branch address onto the stack. Of course if you'll +let me choose the execution location (because I have compromised your +stack) I pretty much own the machine. + +Your fixed code probably does a call to the dynamic library anyway +otherwise it wouldn't have linked in the first place. From your +fixed code I can figure out the branch point your code will use, +compute the offset I want to exploit and push THAT address onto +the stack. Randomizing dynamic library locations is just a trivial +piece of fluff as far as security goes. However it is a huge +performance hit as far as memory management goes. The tradeoff isn't +worth the effort. I have yet to see a convincing argument otherwise. + +>Luckily emacs seems to have sufficient importance that someone else +>will likely do this for us. I've been treating unexec as a black box +>myself. + +Yeah, somebody will figure out unexec. It may even be me (though I +doubt it) :-) + +\start +Date: 02 Dec 2003 11:19:26 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Greetings! + +cvs -d:pserver:anoncvs@subversions.gnu.org:/cvsroot/gcl co -r +Version_2_6_1 -d gcl-2.6.1 gcl + +root writes: + +> Please remind me again of the correct cvs flags to check out the 2.6.1 latest. + +\start +Date: 02 Dec 2003 11:27:17 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Greetings! + +root writes: + +> I've only been attacking the brk randomization issue, not all of +> exec-shield. The claim is that brk randomization will "only" allocate +> within 100M of memory. (We elders-of-the-field consider 1M to be a +> significant amount of memory :-) ) Roland appears to be unmoved by + +Just wanted to point out here that a randomization of this magnitude, +even if gcl can work around it, could potentially result in a loss of +100M in usable memory on each save-system. I think axiom currently +uses about 4 of these. The default memory maximum for GCL is only +128M. Count me among the elders :-). + +> the discussion so I suppose that we may have to fall back on static +> linking as the only acceptable solution. +> + +If you recall, static linking was broken even without exec-shield. I +think if brk randomization remains at the above mentioned range, we +would probably fall back on Roland's linker script solution, if I +understand correctly. + +> The basic idea is that you find an exploit in library code, find out +> where the dynamic library gets loaded, put a bogus "return address" +> onto the stack which branches to the library code and run the exploit. +> Since you don't know where the dynamic library will get loaded you +> can't hard-code a branch address onto the stack. Of course if you'll +> let me choose the execution location (because I have compromised your +> stack) I pretty much own the machine. +> +> Your fixed code probably does a call to the dynamic library anyway +> otherwise it wouldn't have linked in the first place. From your +> fixed code I can figure out the branch point your code will use, +> compute the offset I want to exploit and push THAT address onto +> the stack. Randomizing dynamic library locations is just a trivial +> piece of fluff as far as security goes. However it is a huge +> performance hit as far as memory management goes. The tradeoff isn't +> worth the effort. I have yet to see a convincing argument otherwise. +> + +OK, your understanding is considerably deeper than mine on this +issue. Even granting some security benefit, though, I don't see why +the range can't be quite small and still be effective. How long would +it take someone to guess a randomly generated address in even a 1k +range? + +> >Luckily emacs seems to have sufficient importance that someone else +> >will likely do this for us. I've been treating unexec as a black box +> >myself. +> +> Yeah, somebody will figure out unexec. It may even be me (though I +> doubt it) :-) +> + +\start +Date: Tue, 2 Dec 2003 11:45:55 -0500 +From: root +To: camm@enhanced.com +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +>Just wanted to point out here that a randomization of this magnitude, +>even if gcl can work around it, could potentially result in a loss of +>100M in usable memory on each save-system. I think axiom currently +>uses about 4 of these. The default memory maximum for GCL is only +>128M. Count me among the elders :-). + +Is there any way to get GCL to use all available memory? Fedora will +let you address 4G so Roland doesn't understand why I cry about 100M. + +>If you recall, static linking was broken even without exec-shield. I +>think if brk randomization remains at the above mentioned range, we +>would probably fall back on Roland's linker script solution, if I +>understand correctly. + +I don't understand Roland's linker solution. + +>OK, your understanding is considerably deeper than mine on this +>issue. Even granting some security benefit, though, I don't see why +>the range can't be quite small and still be effective. How long would +>it take someone to guess a randomly generated address in even a 1k +>range? + +The issue is the same at 1k vs 100M but the latter sounds so much more +secure :-) + +\start +Date: Tue, 2 Dec 2003 11:49:34 -0500 +From: root +To: camm@enhanced.com +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +I just got done talking to my son. He is working on a graphics library +that runs in common lisp (was CMUCL, now SBCL). It uses CLX. Does GCL +support CLX? I tried to coerce...umm, convince him to use GCL but he +claims that the ANSI loop macro fails. + +If he gets the graphics library up to snuff we can import the graphics +as lisp code rather than C code. I'm building the C graphics code now +and it is painful. + +\start +Date: 02 Dec 2003 12:11:26 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Greetings! + +root writes: + +> I just got done talking to my son. He is working on a graphics library +> that runs in common lisp (was CMUCL, now SBCL). It uses CLX. Does GCL +> support CLX? I tried to coerce...umm, convince him to use GCL but he + +Last I heard, CLX compiles on GCL out of the box. Would appreciate +reports. + +> claims that the ANSI loop macro fails. +> + +Is he using our ANSI build? I'd appreciate a report here too. We +have an ansi test suite courtesy of Paul Dietz, which gives the loop +macro a heavy workout. Last I checked we were as good as cmucl in +this regard. In the Debian packaged gcl, the test suite results can +be found in /usr/share/doc/{gcl,gclcvs}/test_results.gz. If one wants +ANSI, our gclcvs branch has made some significant improvements over +the stable branch. Just to let you know, though, axiom cannot use the +ansi build until the in-package issue is resolved, at least without +being patched, i.e. externally. Our traditional non-ansi image is +used to build axiom for Debian, as it has the old in-package +behavior. + +> If he gets the graphics library up to snuff we can import the graphics +> as lisp code rather than C code. I'm building the C graphics code now +> and it is painful. +> + +Is this gcl-gtk? We've imported that tree into cvs head, but haven't +worked with it much yet. + +\start +Date: 02 Dec 2003 12:13:10 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Greetings! + +root writes: + +> >Just wanted to point out here that a randomization of this magnitude, +> >even if gcl can work around it, could potentially result in a loss of +> >100M in usable memory on each save-system. I think axiom currently +> >uses about 4 of these. The default memory maximum for GCL is only +> >128M. Count me among the elders :-). +> +> Is there any way to get GCL to use all available memory? Fedora will +> let you address 4G so Roland doesn't understand why I cry about 100M. +> + +Sure, --enable-maxpage should let you go as high as you want. I can't +imagine there not being serious performance issues, but I could be +mistaken. + +> >If you recall, static linking was broken even without exec-shield. I +> >think if brk randomization remains at the above mentioned range, we +> >would probably fall back on Roland's linker script solution, if I +> >understand correctly. +> +> I don't understand Roland's linker solution. +> + +Me either, but sounded promising :-). + +> >OK, your understanding is considerably deeper than mine on this +> >issue. Even granting some security benefit, though, I don't see why +> >the range can't be quite small and still be effective. How long would +> >it take someone to guess a randomly generated address in even a 1k +> >range? +> +> The issue is the same at 1k vs 100M but the latter sounds so much more +> secure :-) +> + +\start +Date: Tue, 2 Dec 2003 14:42:36 -0500 +From: root +To: daly@idsi.net +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] +Cc: camm@enhanced.com + +Camm, + +it appears that the 2-6-1 build has a minor bug. In the final build +line there is a + +cat init_pre_gcl.lsp.tmp | sed \ + +and one of the sed replacement commands reads: + +-e "s,@LI-CC@,\"gcc -c -g -O2 -Wa,--execstack -Wall ...... + ^ + bug? + +It appears that the gcc command generation line has a comma +that should not be there. + +\start +Date: 02 Dec 2003 15:19:02 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: Re: [Axiom-developer] Re: [Gcl-devel] [Re: Executable memory: further programs that fail] + +Thanks! Should be fixed now. + +Take care, + +root writes: + +> Camm, +> +> it appears that the 2-6-1 build has a minor bug. In the final build +> line there is a +> +> cat init_pre_gcl.lsp.tmp | sed \ +> +> and one of the sed replacement commands reads: +> +> -e "s,@LI-CC@,\"gcc -c -g -O2 -Wa,--execstack -Wall ...... +> ^ +> bug? +> +> It appears that the gcc command generation line has a comma +> that should not be there. + +\start +Date: Sun, 7 Dec 2003 01:20:09 +0100 (CET) +From: Bertfried Fauser +To: axiom-developer@nongnu.org +Subject: [Axiom-developer] AXIOM / TeXmacs problems (bugs?) +Cc: axiom@texmacs.org + +Hi, + +I am not quite sure to which mailing list I should send the following. I +am also not sure if some of my findings are bugs or features ... hence I +will provide a few and compile later a longer list which I send after +seeing where to go with. + + +AXIOM (TeXmacs and cmd-line) +1) The .axiom.input file cannot be red by the command + (1)-> )read .axiom + it seem as if AXIOM parses teh filemane and does not recognize such + filenames which are starting with a '.' (Since this file is loaded + automatically its not serious) + WORKAROUND: + (1)-> )read .axiom.input + +2) AXIOM using TeXmacs + In the cmd-line AXIOM I can define functions using several ines of input + where lines which are not the last line are ended by ' _'. In TeXmacs + this confused TeXmacs totally (freezes teh program). I consider this a + bug, since it schould be possible to come up with multi line command + (TeXmacs usesd 'shift RETURN' which might be needed to translated + into ' _' + WORKAROUND: + Use emacs to createand input file foo.input and load it into TeXmascs + via )read foo, however this is not WYSYWIG fashion + +QUESTION: + Should such findings be reportted in teh mailing list or shall I use teh + bug-report form? I have a couple of errors of the open source AXIOM + which was reported to work in the NAG version. + +\start +Date: Sun, 07 Dec 2003 11:20:31 +0100 +From: David MENTRE +To: Bertfried.Fauser@uni-konstanz.de +Subject: [Axiom-developer] Re: AXIOM / TeXmacs problems (bugs?) +Cc: axiom@texmacs.org + +Hello Bertfried, + +Bertfried Fauser writes: + +> Should such findings be reportted in teh mailing list or shall I use teh +> bug-report form? I have a couple of errors of the open source AXIOM +> which was reported to work in the NAG version. + +You should use the bug-report form on Savannah. It helps us not forget +issues. You can however send an email afterwards to inform people of the +issues. + +Savannah is currently down so the mailing-list is the best second +choice. + +\start +Date: Sun, 7 Dec 2003 12:14:45 +0100 (CET) +From: Joris van der Hoeven +To: +Subject: [Axiom-developer] Re: AXIOM / TeXmacs problems (bugs?) +Cc: axiom@texmacs.org + +> 2) AXIOM using TeXmacs +> In the cmd-line AXIOM I can define functions using several ines of input +> where lines which are not the last line are ended by ' _'. In TeXmacs +> this confused TeXmacs totally (freezes teh program). I consider this a +> bug, since it schould be possible to come up with multi line command +> (TeXmacs usesd 'shift RETURN' which might be needed to translated +> into ' _' + +This can indeed be done (the interface for GNUplot does such a thing, +for instance). The maintainers of Axiom interface might wish to take +at the GNUplot interface and do the same thing for Axiom. + +\start +Date: Sun, 07 Dec 2003 13:07:22 +0100 +From: David MENTRE +To: axiom-developer@nongnu.org +Subject: [Axiom-developer] )read .axiom issue (was: Re: AXIOM / TeXmacs problems (bugs?)) + +Hello, + +Bertfried Fauser writes: + +> 1) The .axiom.input file cannot be red by the command +> (1)-> )read .axiom +> it seem as if AXIOM parses teh filemane and does not recognize such +> filenames which are starting with a '.' (Since this file is loaded +> automatically its not serious) +> WORKAROUND: +> (1)-> )read .axiom.input + +Ok. The code that make the )read command is defined in +src/interp/i-syscmd.boot.pamphlet, lines 1406-..., in the readSpad2Cmd +boot function. At first glance, I cannot see what is going wrong. + +Tim, I tried to follow your guidelines to trace this function execution +but the execution of readSpad2Cmd does not stop. Have I missed something? + +(1) -> )trace readSpad2Cmd )break + + Function traced: readSpad2Cmd +(1) -> )read .axiom +1 )trace _$FINDFILE + + Function traced: $FINDFILE +(1) -> )read .axiom +1exit $FINDFILE : NIL + + The file .axiom is needed but does not exist. +(1) -> )read .axiom.input +1exit $FINDFILE : "/home/david/.axiom.input" + + +Is it a Common Lisp issue? + +\start +Date: Mon, 08 Dec 2003 20:00:37 +0100 +From: David MENTRE +To: "Bill Page" +Subject: [Axiom-developer] Re: [Axiom-mail] RE: AXIOM / TeXmacs problems (bugs?) + +[ Continuing on axiom-developer... ] + +Hello, + +"Bill Page" writes: + +> There are differences however even in the semantics when +> you write library routines that are intended to be compiled. +> For example you must provide much more explicit type +> information since you can no longer depend on the interpreter's +> heuristics for type inference. I am not sure how ambitious +> one should be in attempting to resolve such differences. + +Another approach might to use only a compiler, the "interpreter" just +feeding input to the compiler and displaying its output. This is the +approach followed in the Objective Caml language. + +Such a design would raise the issue of having a good type inference +algorithm that would allow (1) a light syntax (an absolut necessity for +the interactive mode) and (2) would be, at the same time, a unique, well +defined and correct type inference. From the few pages I have read in +the Jenks & Sutor book, the "label" type inference (or "record" type +inference) as found in the Objective Caml language and used for the +typing of objects (as in object-oriented) might be sufficient. But +clearly more work would be necessary to have a light syntax in +interactive mode[1]. + +Yours, +d. + +[1] In short, in ML like languages, the type of objects is deduced from + the name of the operator (e.g. '+' has '(INT, INT) -> INT' + signature, using Axiom notation). In Axiom, it appears to me that + typing is more object-oriented like, in the same way as SmallTalk or + C++ with overloading: the type of objects determines the type of + operation to apply. One might envision a typing system where you + have several operators ('+int' for (INT, INT) -> INT, '+poly' for + (POLY, POLY) -> POLY) (thus you can use ML-like type inference) but + hide those several operators with some "hints" from the typing of + previous objects in interactive mode (e.g. x is a POLY, so the '+' + in "x+3" means "use the '+poly'"). I don't know yet if this idea is + just smoke in my brain or a real good idea. :) + +\start +Date: Tue, 9 Dec 2003 11:25:09 -0500 +From: "Bill Page" +To: "'David MENTRE'" +Subject: [Axiom-developer] Axiom interactive input syntax (was: [Axiom-mail] RE: AXIOM / TeXmacs problems(bugs?)) + +On Monday, December 08, 2003 2:01 PM David MENTRE +wrote: + +> +> Another approach might to use only a compiler, the +> "interpreter" just feeding input to the compiler and +> displaying its output. This is the approach followed in +> the Objective Caml language. + +There is also a mode like this in Aldor, the Axiom library +compiler. But compared to the Axiom interpreter it seems +very awkward. Maybe a TeXmacs - Aldor interface would help. +I wonder if anyone has done this yet? I imagine that some +people might already use emacs with Aldor in this way. + +> +> Such a design would raise the issue of having a good type +> inference algorithm that would allow (1) a light syntax +> (an absolute necessity for the interactive mode) and +> (2) would be, at the same time, a unique, well defined and +> correct type inference. + +That is certainly the hard part! + +> From the few pages I have read in the Jenks & Sutor book, +> the "label" type inference (or "record" type inference) as +> found in the Objective Caml language and used for the typing +> of objects (as in object-oriented) might be sufficient. But +> clearly more work would be necessary to have a light syntax +> in interactive mode[1]. + +That is interesting but compared to what we were discussing +originally about unifying the Axiom interactive input language +syntax with the compiler language syntax - this is what I +would call *very* ambitious. + +What I had in mind was much simpler. I thought it might be +useful to modify the TeXmacs/Axiom (tm_axiom) interface +program so that it stores what it gets from TeXmacs (when +you press Enter) into a temporary text file, say +/tmp/texmacs.input, interpreting shift-Enter as new lines, +and then calls Axiom with + + )read /tmp/texmacs.input + +to execute the commands. This would allow the usual indented +block structure style to be used rather than the ( ) block +structure. And the syntax would then be closer to the +compiler syntax and also essentially identical to the code +that is displayed in the Jenks & Sutor book. + +This change is simple so when I have a moment, I will try +it. But I am not sure that it will always work nor whether +it is the best approach. + +> +> Yours, +> d. +> +> [1] In short, in ML like languages, the type of objects is +> deduced from the name of the operator (e.g. '+' has +> '(INT, INT) -> INT' signature, using Axiom notation). In +> Axiom, it appears to me that typing is more object-oriented +> like, in the same way as SmallTalk or C++ with overloading: +> the type of objects determines the type of operation to apply. + +Yes, that is exactly true. Notice that in interactive Axiom +when we define + + f(x) == x+x + +we have not really defined a function yet. It is only a kind +of template for a class of functions. (I am not sure of the +correct technical Axiom term of this.) It is only when I write + + f(2) + +that Axiom compiles a function of type + + Integer -> Integer + +It is during compilation that + is identified as +$Integer, +although I could have been more specific, if I so choose, +when I wrote the function "template" and included more type +information - the way I would have to if I was writing and +compiling a library routine. + +Then when I write + + m := matrix [[1.0,2.0],[3.0,4.0]] + f(m) + +Axiom compiles another function of type + + Matrix Float -> Matrix Float + +etc. So even in an online session we might have several +different functions associated with the overloaded name f. + +> One might envision a typing system where you have several +> operators ('+int' for (INT, INT) -> INT, '+poly' for +> (POLY, POLY) -> POLY) (thus you can use ML-like type +> inference) but hide those several operators with some +> "hints" from the typing of previous objects in interactive +> mode (e.g. x is a POLY, so the '+' in "x+3" means +> "use the '+poly'"). I don't know yet if this idea is +> just smoke in my brain or a real good idea. :) + +Not at all. That is (more or less) exactly how Axiom works +now. See "package calling" in Jenks and Sutor, page 83. + +\start +Date: Tue, 9 Dec 2003 14:32:12 -0500 +From: root +To: bill.page1@sympatico.ca +Subject: [Axiom-developer] Re: [Axiom-math] Bourbaki syndrome, Axiom and OpenMath (was: [om] DefMP elements) + +Bill, + +The openmath library is in the zips subdirectory (OMCv1.4a.tgz) which +I believe was the version that was used with Axiom. + +I spoke to Carlo Traverso who was involved with openmath (I've been +away on a business trip to Italy to talk to Carlo). We'll clearly +have further disucssions on this subject. + +\start +Date: Wed, 10 Dec 2003 08:41:40 +0100 +From: David MENTRE +To: "Bill Page" +Subject: [Axiom-developer] Re: Axiom interactive input syntax +Cc: axiom-developer@nongnu.org + +"Bill Page" writes: + +> On Monday, December 08, 2003 2:01 PM David MENTRE +> wrote: +> +>> (2) would be, at the same time, a unique, well defined and +>> correct type inference. +> +> That is certainly the hard part! + +A hard part, but *maybe* not the most difficult one (if record-like type +inference suits Axiom needs). + +> That is interesting but compared to what we were discussing +> originally about unifying the Axiom interactive input language +> syntax with the compiler language syntax - this is what I +> would call *very* ambitious. + +Yes. :) Never let a computer scientist try to make some science. :) + +> What I had in mind was much simpler. I thought it might be +> useful to modify the TeXmacs/Axiom (tm_axiom) interface +> program so that it stores what it gets from TeXmacs (when +> you press Enter) into a temporary text file, say +> /tmp/texmacs.input, interpreting shift-Enter as new lines, +> and then calls Axiom with +> +> )read /tmp/texmacs.input +> +> to execute the commands. This would allow the usual indented +> block structure style to be used rather than the ( ) block +> structure. And the syntax would then be closer to the +> compiler syntax and also essentially identical to the code +> that is displayed in the Jenks & Sutor book. + +Yes, it might be doable. However I wonder if we want to have a different +syntax between TeXmacs and command-line interfaces. Wouldn't it disturb +the user? More burden for the community? And would the example of the +Axiom book be usable? + +More generally, Axiom is not perfect. We will have to decide at which +point we make more noise and start its promotion. (I think having the +freely available Axiom book is a requirement) We will probably then have +a more important user community and changes will be more difficult +(due to user habits). I have not predefined answer on such issues. + + +>> One might envision a typing system where you have several +>> operators ('+int' for (INT, INT) -> INT, '+poly' for +>> (POLY, POLY) -> POLY) (thus you can use ML-like type +>> inference) but hide those several operators with some +>> "hints" from the typing of previous objects in interactive +>> mode (e.g. x is a POLY, so the '+' in "x+3" means +>> "use the '+poly'"). I don't know yet if this idea is +>> just smoke in my brain or a real good idea. :) +> +> Not at all. That is (more or less) exactly how Axiom works +> now. See "package calling" in Jenks and Sutor, page 83. + +I'm only page 69. :) + +In any case, many thanks for the feedback. + +\start +Date: Wed, 10 Dec 2003 12:15:24 +0100 (CET) +From: Bertfried Fauser +To: David MENTRE +Subject: Re: [Axiom-developer] Re: Axiom interactive input syntax +Cc: Bill Page + +On Wed, 10 Dec 2003, David MENTRE wrote: + +Dear Bill and David, + +a very interesting discussion. As a prototype of an ignorant user, I would +be satisfied by a clear statement of what kind of input is necessary in +what interface. Just to tp`ype a few ' _' signs in a comand line session +is no trouble. But I hadn't found no hint that the syntax is different. +That was the most confusing point and that is most easily cured I think. + + +> >> (2) would be, at the same time, a unique, well defined and +> >> correct type inference. +> > +> > That is certainly the hard part! +> +> A hard part, but *maybe* not the most difficult one (if record-like type +> inference suits Axiom needs). +> +> > That is interesting but compared to what we were discussing +> > originally about unifying the Axiom interactive input language +> > syntax with the compiler language syntax - this is what I +> > would call *very* ambitious. +> +> Yes. :) Never let a computer scientist try to make some science. :) + +Why NOT? Or was all of AXIOMS algebra coded by mathematicians? I doubt +it... + +However, I do *not* see any possibility to detect automatically the type +of a data structure. Hence AXIOM will for sure fail in some cases to +compile a correct function if no type specification is given. +Eg: A list of intergers could be a list of bases in a Grassmann algebra +*or* a partition *or* a .... if I would like to add these, I need +'+GRASS' and '+PART' and ... and AXIOM cannot decide this. In this sense, +a 'soft' user interface would need to be even inquisitive about the user +and ask interactively about data types if ther eis more than one known +choice for a libary function. + Even worse if you define functions, which might implement new such +possibilities and the compiler would even need to *understand* the +legality of a proceedure on a certain data structure. That's even a task +difficult for mathematicians. + +What would be of utmost help to me would be a very very good graphical +type brouser. Indeed even with the large amount of series one is +tourtored. Furthermore its not so easy to convert types and there should +be when ever possible a cast operator to perform such changes. At least in +such a direction to the more general ttype, hence loosing information. + Eg a Euclidean ring is also a ring and if Euclidean is not +necesary or even disturbe then it might be dropped. IF later the property +Euclidean of that data is needed, AXIOM is lost, since checking for such a +property might be impossible without further information or user help. + A brouser could help to keep trak of system wide known typse and +might come up with a dependence structure (like the algebra dependencies?) + +\start +Date: Wed, 10 Dec 2003 10:20:50 +0000 +From: nic +To: Bertfried.Fauser@uni-konstanz.de +Subject: Re: [Axiom-developer] Re: Axiom interactive input syntax +Cc: Bill Page + +On Wed, 2003-12-10 at 11:15, Bertfried Fauser wrote: +> What would be of utmost help to me would be a very very good graphical +> type brouser. Indeed even with the large amount of series one is +> tourtored. Furthermore its not so easy to convert types and there should +> be when ever possible a cast operator to perform such changes. At least in +> such a direction to the more general ttype, hence loosing information. +> Eg a Euclidean ring is also a ring and if Euclidean is not +> necesary or even disturbe then it might be dropped. IF later the property +> Euclidean of that data is needed, AXIOM is lost, since checking for such a +> property might be impossible without further information or user help. +> A brouser could help to keep trak of system wide known typse and +> might come up with a dependence structure (like the algebra dependencies?) + +Excuse me if I've got the wrong end of the stick. + +I did some work on automatic type changing in the interpreter for my +thesis under James Davenport... + +I should have the BOOT code at home on CD somewhere, although I did have +some problems integrating the final version into the interpreter. + +You can get the thesis from http://www.nic.uklinux.net/research/phd.ps + +nic + +(PS. Hi to Mike Dewar and anyone else I knew back then). +-- +nic + +\start +Date: 10 Dec 2003 10:03:14 -0500 +To: "Vadim V. Zhytnikov" +From: Camm Maguire +Subject: [Axiom-developer] Re: [Gcl-devel] GCL on mingw +Cc: Mike Thomas + +Hi Vadim! + +"Vadim V. Zhytnikov" writes: + +> Hi! +> +> Trying to build recent GCL 2.6.1 on mingw I encountered + + ^^^^^^^^^^^^^^^ + +Great! I was just going to ask you if you had a mingw development +system to work with given your earlier mingw problem report. + +> some strange problem on configure stage. +> The following test fails +> =========================================================== +> echo $ac_n "checking sizeof struct contblock""... $ac_c" 1>&6 +> echo "configure:3238: checking sizeof struct contblock" >&5 +> if test "$cross_compiling" = yes; then +> echo Cannot find sizeof struct contblock;exit 1 +> else +> cat > conftest.$ac_ext < #line 3243 "configure" +> #include "confdefs.h" +> #include +> #define EXTER +> #include "$MP_INCLUDE" +> #include "`pwd`/h/enum.h" +> #include "`pwd`/h/object.h" +> int main(int argc,char **argv,char **envp) { +> FILE *f=fopen("conftest1","w"); +> fprintf(f,"%u",sizeof(struct contblock)); +> fclose(f); +> return 0; +> } +> EOF +> =========================================================== +> Trouble makes are these two lines +> #include "`pwd`/h/enum.h" +> #include "`pwd`/h/object.h" +> Due to some reason under mingw +> #include "/home/vadim/gcl/h/enum.h" +> signals an error: File not found. +> I really don't understand such strange behavior +> since ls /home/vadim/gcl/h/enum.h works fine. +> But maybe we just can replace these two lines by +> #include "h/enum.h" +> #include "h/object.h" + +I'll look into making this change. Unfortunately, write access to cvs +at savannah is still down. Please remind me if I forget by the time +it is restored. + +> +> With such modification I was able to build +> ANSI GCL on mingw. My goal was to test strange + + ^^^^ + +Really? You built pcl? We definitely need the details here if so, as +Mike has experienced problems getting through this stage, and has had +to use precompiled .c source to ship his binary ansi package. + +BTW, I do suspect the problem you report and Mike's build problem +stems from the same source. + +> memory-related GCL crashes under mingw. +> I tried various memory allocation tests - +> exactly the same I used on Linux (see e.g. +> atest.lisp in attachment). In general +> results are practically the same on both +> platforms with one important exception. +> While on Linux I maximally can use 110K pages +> (MAXPAGES=128K) on mingw all attempts +> to allocate more than ~62000 pages +> causes allocation error. GCL self terminates +> with the message: +> Unrecoverable error: Can't allocate +> + +My suspicion is that the heap is growing into some memory area already +in use for something else, e.g. shared libs. I think Mike is away at +the moment, but I had previously requested the following from him, +which you may now be able to provide for me: + +1) value of the configure determined define DBEGIN +2) on building a gcl with --enable-debug, run under gdb, breaking at + main, and report the value of 'p sbrk(0)' +3) break at init_lisp, stop at this line: + + if (NULL_OR_ON_C_STACK(&j) == 0 + || NULL_OR_ON_C_STACK(Cnil) != 0 + || (((unsigned long )core_end) !=0 + && NULL_OR_ON_C_STACK(core_end) != 0)) + { /* check person has correct definition of above */ + error("NULL_OR_ON_C_STACK macro invalid"); + } + + and report the values returned by 'p &j', 'p &Cnil_body', and 'p + core_end'. + +4) Try to let me know if the C stack counts up or down. I.e. break in + some function with a local variable defined, and print the address + of that variable, and compare it to the address of a local variable + defined in a surrounding function (i.e. a parent function). + +We have the following somewhat less than robust code currently in +place for MINGW (main.c): + +#ifdef _WIN32 +unsigned int _dbegin = 0x10100000; +unsigned int _stacktop, _stackbottom; +#endif + +#ifdef _WIN32 + { + unsigned int dummy; + + _stackbottom = (unsigned int ) &dummy; + _stacktop = _stackbottom - 0x10000; // ??? + + } +#endif + + +So from this, sbrk(0) should begin at around 0x10100000, and the stack +should count down from some unknown (to me at least) address region. +Please try to verify this and fill in the holes. It would be great to +firm this up, particularly the hardcoded stack area limit. + +Then, if Mingw has some analog of ldd or /proc/$pid/maps, please +report their contents/output to me on the image running under gdb. +I.e. 'ldd saved_gcl' and 'cat /proc/(process id of saved_gcl)/maps'. + +I'm assuming the error message you saw was: + + IF_ALLOCATE_ERR error("Can't allocate. Good-bye!"); + +(There are a few other error messages beginning with 'Can't +allocate'). If so, my guess is that sbrk has hit a large jump. We +have another somewhat ad hoc piece of code in place for mingw at +present (mingw.h): + +#define IF_ALLOCATE_ERR \ + if (core_end != sbrk(0))\ + {char * e = sbrk(0); \ + if (e - core_end < 0x10000 ) { \ + int i; \ + for (i=page(core_end); i < page(e); i++) { \ + type_map[i] = t_other; \ + } \ + core_end = e; \ + } \ + else \ + error("Someone allocated my memory!");} \ + if (core_end != (sbrk(PAGESIZE*(n - m)))) + + + +The analog for linux (bsd.h): + +#define ROUND_UP_SBRK(x) \ + do {long i; \ + if ((i = ((long)x & (PAGESIZE - 1)))) \ + x=sbrk(PAGESIZE - i); } while(0); + +#define FIX_RANDOM_SBRK \ +do {char *x=sbrk(0); \ + if (core_end != x) \ + { ROUND_UP_SBRK(x); x=sbrk(0);\ + while (core_end < x) \ + { type_map[page(core_end)]= t_other; \ + core_end = core_end + PAGESIZE;} \ + if (core_end !=x) error("Someone allocated my memory");}} while (0) + + +#define IF_ALLOCATE_ERR \ + FIX_RANDOM_SBRK; \ + if (core_end != sbrk(PAGESIZE*(n - m))) + + +has no prescribed limit of 0x10000. Mike, where does this come from? + +As a bonus, examining this code leads me to suspect that we already +have designed in mechanims to handle non-contiguous sbrk, a la +exec-shield, meaning that it is likely that someone has made sure GCL +would work under an exec-shield like randomized sbrk, barring unexec +problems as earlier discussed. + +This doesn't yet address your other post, where there is no "Can't +allocate" error, but gives an important clue, I feel. + + +Take care, + +> At present I don't know why this happens. +> Any ideas? +> +> +> +> -- +> Vadim V. Zhytnikov +> +> +> +> (si::allocate-growth 'cons 1 1000 66 33) +> (setq cnt 0) +> (si::gbc-time 0) +> (setq w nil) +> +> (defun pass () +> (progn (setq cnt (1+ cnt)) +> (format t "***** Starting pass #~d" cnt) +> (time(setq w (cons (make-list 3000000) w))) +> (room) +> (format t "***** End of pass #~d" cnt) (terpri) +> (format t "***** Run time: ~,2F GC time: ~,2F (~,1F%)" +> (/ (get-internal-run-time) 100.0) +> (/ (si::gbc-time) 100.0) +> (* (/ (si::gbc-time) (get-internal-run-time)) 100.0)) +> )) + +\start +Date: Wed, 10 Dec 2003 11:26:34 -0500 +From: "Bill Page" +To: "'nic'" +Subject: RE: [Axiom-developer] Re: Axiom interactive input syntax + +On Wednesday, December 10, 2003 5:21 AM nic +[mailto:nic@uklinux.net] wrote: + +> ... +> Excuse me if I've got the wrong end of the stick. +> + +Not at all, you have come to the right place! + +> I did some work on automatic type changing in the +> interpreter for my thesis under James Davenport... +> +> I should have the BOOT code at home on CD somewhere, although +> I did have some problems integrating the final version into +> the interpreter. +> +> You can get the thesis from +> http://www.nic.uklinux.net/research/phd.ps +> + +Thank you very much indeed for this contribution. I have +so far only just quickly skimmed your thesis but it appears +to me to be *exactly* on the topic that we are discussing. +And it seems "ambitious" in exactly the kind of way that +David Mentre outlined earlier. + +I personally am very highly motivated to implement and test +the kind of changes to Axiom that you describe in your thesis. +Now that Axiom is open source and we have a small (but growing) +body of Axiom expertise available here, I think it would be +great to carry out the kind of project that your thesis +initiates. + +Tim, can we make some arrangement, e.g. a new CVS development +branch where Nic could upload his modified BOOT code and +we could proceed with a new experimental version of Axiom +without interfering with the ongoing process of releasing +more of the original Axiom code. + +\start +Date: Wed, 10 Dec 2003 20:30:20 +0300 +From: "Vadim V. Zhytnikov" +To: Camm Maguire +Subject: [Axiom-developer] Re: [Gcl-devel] GCL on mingw +Cc: Mike Thomas + +Camm Maguire ?????: + +> Hi Vadim! +> +> "Vadim V. Zhytnikov" writes: +> +> +>>Hi! +>> +>>Trying to build recent GCL 2.6.1 on mingw I encountered +> +> +> ^^^^^^^^^^^^^^^ +> +> Great! I was just going to ask you if you had a mingw development +> system to work with given your earlier mingw problem report. +> + +Well, I just followed Mike's readme.mingw instruction. +The key point is additional (si::use-fast-links nil) in +pcl/makefile. He also apparently recommends --enable-custreloc +but I was able to build ANSI GCL with and without this option. +On the other hand I did it with some gcl 2.6.1 snapshot not +with very last CVS sources - I have to try it once again. + +> +>>some strange problem on configure stage. +>>The following test fails +>>=========================================================== +>>echo $ac_n "checking sizeof struct contblock""... $ac_c" 1>&6 +>>echo "configure:3238: checking sizeof struct contblock" >&5 +>>if test "$cross_compiling" = yes; then +>> echo Cannot find sizeof struct contblock;exit 1 +>>else +>> cat > conftest.$ac_ext <>#line 3243 "configure" +>>#include "confdefs.h" +>>#include +>> #define EXTER +>> #include "$MP_INCLUDE" +>> #include "`pwd`/h/enum.h" +>> #include "`pwd`/h/object.h" +>> int main(int argc,char **argv,char **envp) { +>> FILE *f=fopen("conftest1","w"); +>> fprintf(f,"%u",sizeof(struct contblock)); +>> fclose(f); +>> return 0; +>> } +>>EOF +>>=========================================================== +>>Trouble makes are these two lines +>> #include "`pwd`/h/enum.h" +>> #include "`pwd`/h/object.h" +>>Due to some reason under mingw +>> #include "/home/vadim/gcl/h/enum.h" +>>signals an error: File not found. +>>I really don't understand such strange behavior +>>since ls /home/vadim/gcl/h/enum.h works fine. +>>But maybe we just can replace these two lines by +>> #include "h/enum.h" +>> #include "h/object.h" +> +> +> I'll look into making this change. Unfortunately, write access to cvs +> at savannah is still down. Please remind me if I forget by the time +> it is restored. +> +> +>>With such modification I was able to build +>>ANSI GCL on mingw. My goal was to test strange +> +> +> ^^^^ +> +> Really? You built pcl? We definitely need the details here if so, as +> Mike has experienced problems getting through this stage, and has had +> to use precompiled .c source to ship his binary ansi package. +> +> BTW, I do suspect the problem you report and Mike's build problem +> stems from the same source. +> + +See above. + +> +>>memory-related GCL crashes under mingw. +>>I tried various memory allocation tests - +>>exactly the same I used on Linux (see e.g. +>>atest.lisp in attachment). In general +>>results are practically the same on both +>>platforms with one important exception. +>>While on Linux I maximally can use 110K pages +>>(MAXPAGES=128K) on mingw all attempts +>>to allocate more than ~62000 pages +>>causes allocation error. GCL self terminates +>>with the message: +>>Unrecoverable error: Can't allocate +>> +> +> +> My suspicion is that the heap is growing into some memory area already +> in use for something else, e.g. shared libs. I think Mike is away at +> the moment, but I had previously requested the following from him, +> which you may now be able to provide for me: +> +> 1) value of the configure determined define DBEGIN +> 2) on building a gcl with --enable-debug, run under gdb, breaking at +> main, and report the value of 'p sbrk(0)' +> 3) break at init_lisp, stop at this line: +> +> if (NULL_OR_ON_C_STACK(&j) == 0 +> || NULL_OR_ON_C_STACK(Cnil) != 0 +> || (((unsigned long )core_end) !=0 +> && NULL_OR_ON_C_STACK(core_end) != 0)) +> { /* check person has correct definition of above */ +> error("NULL_OR_ON_C_STACK macro invalid"); +> } +> +> and report the values returned by 'p &j', 'p &Cnil_body', and 'p +> core_end'. +> +> 4) Try to let me know if the C stack counts up or down. I.e. break in +> some function with a local variable defined, and print the address +> of that variable, and compare it to the address of a local variable +> defined in a surrounding function (i.e. a parent function). +> +> We have the following somewhat less than robust code currently in +> place for MINGW (main.c): +> +> #ifdef _WIN32 +> unsigned int _dbegin = 0x10100000; +> unsigned int _stacktop, _stackbottom; +> #endif +> +> #ifdef _WIN32 +> { +> unsigned int dummy; +> +> _stackbottom = (unsigned int ) &dummy; +> _stacktop = _stackbottom - 0x10000; // ??? +> +> } +> #endif +> +> +> So from this, sbrk(0) should begin at around 0x10100000, and the stack +> should count down from some unknown (to me at least) address region. +> Please try to verify this and fill in the holes. It would be great to +> firm this up, particularly the hardcoded stack area limit. +> +> Then, if Mingw has some analog of ldd or /proc/$pid/maps, please +> report their contents/output to me on the image running under gdb. +> I.e. 'ldd saved_gcl' and 'cat /proc/(process id of saved_gcl)/maps'. + +As far as I know there is no analog of /proc/../maps on mingw. + +> +> I'm assuming the error message you saw was: +> +> IF_ALLOCATE_ERR error("Can't allocate. Good-bye!"); +> + +Right. + +> (There are a few other error messages beginning with 'Can't +> allocate'). If so, my guess is that sbrk has hit a large jump. We +> have another somewhat ad hoc piece of code in place for mingw at +> present (mingw.h): +> +> #define IF_ALLOCATE_ERR \ +> if (core_end != sbrk(0))\ +> {char * e = sbrk(0); \ +> if (e - core_end < 0x10000 ) { \ +> int i; \ +> for (i=page(core_end); i < page(e); i++) { \ +> type_map[i] = t_other; \ +> } \ +> core_end = e; \ +> } \ +> else \ +> error("Someone allocated my memory!");} \ +> if (core_end != (sbrk(PAGESIZE*(n - m)))) +> +> +> +> The analog for linux (bsd.h): +> +> #define ROUND_UP_SBRK(x) \ +> do {long i; \ +> if ((i = ((long)x & (PAGESIZE - 1)))) \ +> x=sbrk(PAGESIZE - i); } while(0); +> +> #define FIX_RANDOM_SBRK \ +> do {char *x=sbrk(0); \ +> if (core_end != x) \ +> { ROUND_UP_SBRK(x); x=sbrk(0);\ +> while (core_end < x) \ +> { type_map[page(core_end)]= t_other; \ +> core_end = core_end + PAGESIZE;} \ +> if (core_end !=x) error("Someone allocated my memory");}} while (0) +> +> +> #define IF_ALLOCATE_ERR \ +> FIX_RANDOM_SBRK; \ +> if (core_end != sbrk(PAGESIZE*(n - m))) +> +> +> has no prescribed limit of 0x10000. Mike, where does this come from? +> +> As a bonus, examining this code leads me to suspect that we already +> have designed in mechanims to handle non-contiguous sbrk, a la +> exec-shield, meaning that it is likely that someone has made sure GCL +> would work under an exec-shield like randomized sbrk, barring unexec +> problems as earlier discussed. +> +> This doesn't yet address your other post, where there is no "Can't +> allocate" error, but gives an important clue, I feel. +> +> +> Take care, +> +> + +I'll be able to do tests you suggest this weekend. +At present I just recompiled GCL with 256K maxpages +but nothing changed. + +\start +Date: Wed, 10 Dec 2003 13:06:22 -0500 +From: Tim Daly +To: bill.page1@sympatico.ca, nic@uklinux.net, bertfried.fauser@uni-konstanz.de +Subject: [Axiom-developer] Axiom interactive input syntax +Cc: axiom-developer@nongnu.org, daly@idsi.net, axiom-mail@nongnu.org + +I've downloaded the thesis and quickly browsed it. +This is an excellent piece of work. It appears that a "real" +implementation ran aground on the rocks of "getdatabase" +(my code :-( ). I think we should definitely consider implementing +this code, adding the extensions, and playing with the result. + +I've inquired about a way to set up a "research" tree for axiom +on savannah. I'm unaware of how to do this using cvs. I'd like to +be able to incant: + +cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom co research + +does anyone know how to set up a new root? + +\start +Date: Wed, 10 Dec 2003 21:22:09 +0100 +From: David MENTRE +To: Bertfried.Fauser@uni-konstanz.de +Subject: Re: [Axiom-developer] Re: Axiom interactive input syntax + +Hello Bertfried, + +Bertfried Fauser writes: + +> What would be of utmost help to me would be a very very good graphical +> type brouser. + +Could you elaborate? What kind of behaviour would you like to have? + +\start +Date: Wed, 10 Dec 2003 22:19:27 +0100 +From: David MENTRE +To: nic +Subject: Type coercicion (was: Re: [Axiom-developer] Re: Axiom interactive input syntax) + +Hello, + +nic writes: + +> I did some work on automatic type changing in the interpreter for my +> thesis under James Davenport... + +>From what I have understood from the introduction and conclusion, you +have an algorithm to transform any "type"[1] in any other mathematically +compatible "type" (provided the type system follows some mathematical +guidelines). This makes me ask the following naïve and meta question: +how would you use such an algorithm? From user behavior (tim-user uses +variable x with another operator implying a new type context) or with +explicit user information (calls to coerce)? Within a type inference +algorithm? + +Yours, +david + +[1] Sorry to probably not use the proper term. + +\start +Date: 10 Dec 2003 16:58:45 -0500 +From: Camm Maguire +To: "Vadim V. Zhytnikov" +Subject: [Axiom-developer] Re: [Gcl-devel] GCL on mingw +Cc: Mike Thomas + +Greetings! + +"Vadim V. Zhytnikov" writes: + +> Camm Maguire ?????: +> +> > Hi Vadim! +> > "Vadim V. Zhytnikov" writes: +> > +> >>Hi! +> >> +> >>Trying to build recent GCL 2.6.1 on mingw I encountered +> > ^^^^^^^^^^^^^^^ +> > Great! I was just going to ask you if you had a mingw development +> > system to work with given your earlier mingw problem report. +> > +> +> Well, I just followed Mike's readme.mingw instruction. +> The key point is additional (si::use-fast-links nil) in + + ^^^^^^^^^^^^^^^^^^^^^^^ + +Thanks for the reminder. I had forgotten about this. I'm wondering +if this is the only place where fast-links cause a problem. If so, it +may simply be in the large array GCL allocates for the purposes of +toggling between fast and slow function pointers coupled with the +memory allocation issues you are observing.. If not, perhaps there is +some alignment issue, or (much worse), some ia64-like retrictions on +function calls via function pointers when the shared library maps +could change across runs. We should be able to break at the bad +function jump in pcl_braid.c (see earlier thread with the debugging +details if interested), go up one frame, disassemble, and look at the +registers to see if the function address has somehow been corrupted. + +> pcl/makefile. He also apparently recommends --enable-custreloc +> but I was able to build ANSI GCL with and without this option. + +??? Meaning via statsysbfd (the default), or via dlopen? + +> On the other hand I did it with some gcl 2.6.1 snapshot not +> with very last CVS sources - I have to try it once again. +> + +It would be nice if you could also verify the ansi build crash when +not turning off fast-links and building without custreloc. + +> > +> >>some strange problem on configure stage. +> >>The following test fails +> >>=========================================================== +> >>echo $ac_n "checking sizeof struct contblock""... $ac_c" 1>&6 +> >>echo "configure:3238: checking sizeof struct contblock" >&5 +> >>if test "$cross_compiling" = yes; then +> >> echo Cannot find sizeof struct contblock;exit 1 +> >>else +> >> cat > conftest.$ac_ext < >>#line 3243 "configure" +> >>#include "confdefs.h" +> >>#include +> >> #define EXTER +> >> #include "$MP_INCLUDE" +> >> #include "`pwd`/h/enum.h" +> >> #include "`pwd`/h/object.h" +> >> int main(int argc,char **argv,char **envp) { +> >> FILE *f=fopen("conftest1","w"); +> >> fprintf(f,"%u",sizeof(struct contblock)); +> >> fclose(f); +> >> return 0; +> >> } +> >>EOF +> >>=========================================================== +> >>Trouble makes are these two lines +> >> #include "`pwd`/h/enum.h" +> >> #include "`pwd`/h/object.h" +> >>Due to some reason under mingw +> >> #include "/home/vadim/gcl/h/enum.h" +> >>signals an error: File not found. +> >>I really don't understand such strange behavior +> >>since ls /home/vadim/gcl/h/enum.h works fine. +> >>But maybe we just can replace these two lines by +> >> #include "h/enum.h" +> >> #include "h/object.h" +> > I'll look into making this change. Unfortunately, write access to +> > cvs +> > at savannah is still down. Please remind me if I forget by the time +> > it is restored. +> >>With such modification I was able to build +> >>ANSI GCL on mingw. My goal was to test strange +> > ^^^^ +> > Really? You built pcl? We definitely need the details here if so, +> > as +> > Mike has experienced problems getting through this stage, and has had +> > to use precompiled .c source to ship his binary ansi package. +> > BTW, I do suspect the problem you report and Mike's build problem +> > stems from the same source. +> > +> +> See above. +> +> > +> >>memory-related GCL crashes under mingw. +> >>I tried various memory allocation tests - +> >>exactly the same I used on Linux (see e.g. +> >>atest.lisp in attachment). In general +> >>results are practically the same on both +> >>platforms with one important exception. +> >>While on Linux I maximally can use 110K pages +> >>(MAXPAGES=128K) on mingw all attempts +> >>to allocate more than ~62000 pages +> >>causes allocation error. GCL self terminates +> >>with the message: +> >>Unrecoverable error: Can't allocate +> >> +> > My suspicion is that the heap is growing into some memory area +> > already +> > in use for something else, e.g. shared libs. I think Mike is away at +> > the moment, but I had previously requested the following from him, +> > which you may now be able to provide for me: +> > 1) value of the configure determined define DBEGIN +> > 2) on building a gcl with --enable-debug, run under gdb, breaking at +> > main, and report the value of 'p sbrk(0)' +> > 3) break at init_lisp, stop at this line: +> > if (NULL_OR_ON_C_STACK(&j) == 0 +> > || NULL_OR_ON_C_STACK(Cnil) != 0 +> > || (((unsigned long )core_end) !=0 +> > && NULL_OR_ON_C_STACK(core_end) != 0)) +> > { /* check person has correct definition of above */ +> > error("NULL_OR_ON_C_STACK macro invalid"); +> > } +> > and report the values returned by 'p &j', 'p &Cnil_body', and 'p +> > core_end'. +> > 4) Try to let me know if the C stack counts up or down. I.e. break +> > in +> > some function with a local variable defined, and print the address +> > of that variable, and compare it to the address of a local variable +> > defined in a surrounding function (i.e. a parent function). +> > We have the following somewhat less than robust code currently in +> > place for MINGW (main.c): +> > #ifdef _WIN32 +> > unsigned int _dbegin = 0x10100000; +> > unsigned int _stacktop, _stackbottom; +> > #endif +> > #ifdef _WIN32 +> > { +> > unsigned int dummy; +> > _stackbottom = (unsigned int ) &dummy; +> > _stacktop = _stackbottom - 0x10000; // ??? +> > } +> > #endif +> > So from this, sbrk(0) should begin at around 0x10100000, and the +> > stack +> > should count down from some unknown (to me at least) address region. +> > Please try to verify this and fill in the holes. It would be great to +> > firm this up, particularly the hardcoded stack area limit. +> > Then, if Mingw has some analog of ldd or /proc/$pid/maps, please +> > report their contents/output to me on the image running under gdb. +> > I.e. 'ldd saved_gcl' and 'cat /proc/(process id of saved_gcl)/maps'. +> +> As far as I know there is no analog of /proc/../maps on mingw. +> +> > I'm assuming the error message you saw was: +> > IF_ALLOCATE_ERR error("Can't allocate. Good-bye!"); +> > +> +> Right. +> +> > (There are a few other error messages beginning with 'Can't +> > allocate'). If so, my guess is that sbrk has hit a large jump. We +> > have another somewhat ad hoc piece of code in place for mingw at +> > present (mingw.h): +> > #define IF_ALLOCATE_ERR \ +> > if (core_end != sbrk(0))\ +> > {char * e = sbrk(0); \ +> > if (e - core_end < 0x10000 ) { \ +> > int i; \ +> > for (i=page(core_end); i < page(e); i++) { \ +> > type_map[i] = t_other; \ +> > } \ +> > core_end = e; \ +> > } \ +> > else \ +> > error("Someone allocated my memory!");} \ +> > if (core_end != (sbrk(PAGESIZE*(n - m)))) +> > The analog for linux (bsd.h): +> > #define ROUND_UP_SBRK(x) \ +> > do {long i; \ +> > if ((i = ((long)x & (PAGESIZE - 1)))) \ +> > x=sbrk(PAGESIZE - i); } while(0); +> > #define FIX_RANDOM_SBRK \ +> > do {char *x=sbrk(0); \ +> > if (core_end != x) \ +> > { ROUND_UP_SBRK(x); x=sbrk(0);\ +> > while (core_end < x) \ +> > { type_map[page(core_end)]= t_other; \ +> > core_end = core_end + PAGESIZE;} \ +> > if (core_end !=x) error("Someone allocated my memory");}} while (0) +> > #define IF_ALLOCATE_ERR \ +> > FIX_RANDOM_SBRK; \ +> > if (core_end != sbrk(PAGESIZE*(n - m))) +> > has no prescribed limit of 0x10000. Mike, where does this come from? +> > As a bonus, examining this code leads me to suspect that we already +> > have designed in mechanims to handle non-contiguous sbrk, a la +> > exec-shield, meaning that it is likely that someone has made sure GCL +> > would work under an exec-shield like randomized sbrk, barring unexec +> > problems as earlier discussed. +> > This doesn't yet address your other post, where there is no "Can't +> > allocate" error, but gives an important clue, I feel. +> > Take care, +> > +> +> I'll be able to do tests you suggest this weekend. +> At present I just recompiled GCL with 256K maxpages +> but nothing changed. +> + +Which would be consistent with the theory that something is occupying +memory at a fixed offset above DBEGIN into which the heap would like +to grow. + +\start +Date: Thu, 11 Dec 2003 11:50:01 +0100 (CET) +From: Bertfried Fauser +To: David MENTRE +Subject: Re: [Axiom-developer] Re: Axiom interactive input syntax + +On Wed, 10 Dec 2003, David MENTRE wrote: + +> > What would be of utmost help to me would be a very very good graphical +> > type brouser. +> +> Could you elaborate? What kind of behaviour would you like to have? + +Dear David, + + perhaps I am describing what the AXIOM browser will do nayway. +However, for me AXIOM is still lacking a f`type browser which can +graphically (like a file system tree) show the dependencies of typs. + +Eg (fictious I have currently no AXIOM available) + + browse Ring + + Ring + |___ Euclidean Ring + | |____ Division Ring + |___ Artinear Ring + |... + ... + +which shows which types are subtypes, (downsearch) + +A second way of working should diplay the upbraces of teh type tree +(graph?) so that you see all datastructures which contain the type +Ring, that may be modules, groups, abelian monoids, etc.... + +This type of knowledge is importand to program AXIOM packages. I spent +currently the most time with AXIOM by searching correct types (in fact the +most general type which allows me to do teh calculation I want to perform) + +I havn't looked into this, but I think the algebra dependen`cies will reflect +somehow the type structure, maybe I am wrong. + +\start +Date: Thu, 11 Dec 2003 12:30:42 -0500 +From: Tim Daly +To: nic@uklinux.net +Subject: [Axiom-developer] Re: phd thesis +Cc: bill.page1@sympatico.ca, Bertfried.Fauser@uni-konstanz.de + +Nicolas, + +We've discussed setting up a "research" branch of axiom so we can +experiment with new ideas without breaking the build for other users. +I expect there will be several "research" branches because I know of +others who want to experiment with axiom. Indeed, axiom's basic +strength is that it is a very good platform for research. + +re: fedora core + +The build will almost work fine on Fedora (I'm also a fedora developer +and have been campaigning against the change that breaks axiom). In +order to get around the problem you need to become root and type: + +echo 0 >/proc/sys/kernel/exec-shield + +Once that is done you can correctly build Axiom as a regular user. +In theory you can download Axiom by incanting: + +cd (yourpath) +cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom login +cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom co axiom +cd axiom +export AXIOM=(yourpath)/axiom/mnt/linux +make + +When prompted for a password just hit enter. + +However, the gnu website is down due to an audit. + + +I've uploaded a very late (possibly not the latest) version of +Axiom to my own website. You can get it and build it by: + +cd (yourpath) +visit axiom.tenken.org +download the "20031211 sources" +tar -zxf axiom.20031211.tgz +cd axiom +export AXIOM=(yourpath)/axiom/mnt/linux +make + +re: solaris, etc + +>I also have access to Red Hat Enterprise Linux 2.1 and 3 boxen as well +>as Solaris 8 if there's any porting help needed. I can also build +>RPMs... + +The other boxen are interesting, particularly the solaris 8 version. +I've tried to download solaris86 to do a build but that whole process +failed and I'll have to try again. If you want to try a port you +should only need to modify the top level makefile but you'll almost +certainly need my help with that. + +re: boot code, thesis, etc + +> Yes I hope so! +>It's not very well written as I was more of a Mathematician than a coder +>at the time and I couldn't get the final version to integrate into the +>interpreter (although a previous, lost version had integrated +>perfectly). + +not to worry. the code that lives in axiom was never expected to be +released to public view. in fact a lot of the interpreter code is mine +and it is both bad and undocumented. + +>Yes. I'm pro-open source (I prefer "Free" software, but open source is +>a good compromise). + +umm, yeah, we've had endless discussions about this and all discussion +is now "banned" from the axiom-developer mailing list. I've created an +axiom-legal mailing list just for that purpose. I have email from +Stallman "blessing" the work as "ok". I had no choice about the +modified BSD license so you'll have to flame Mike Dewar about it. + +(On the savannah.nongnu.org/projects/axiom website you can sign up +to join the various mailing lists) + +>> 3) would you be willing to allow a modified version of your thesis to +>> be used in a "literate program" style to document that code? + +>Yes. Of course! + +I (hope, presume) your thesis is in latex. I'd hate to have to retype it. +Literate programs are basically latex code with 2 additional "tags". +You write a document: + +\documentclass{... +\begin{document} +\chapter{... +..... + +Then you use a chunk definition tag which begins with '<<', contains +any string, and ends with '>>='. Text starting after this tag is +quoted, collected up, and put in a hash table. The chunk ends when +it finds an '@' in column 1. Thus: + +\section +lots of good comments +<>= + your code goes here +@ +\section + +Later you can reference the chunk by referencing the name thus: + +<> + +There is a "default" chunk name which will get expanded automatically +called '<<*>>=' thus: + +<<*>>= +<> +<> +<> +@ + +\end{document} + +So, suppose you save the pile above as a "literate program". +I refer to them as "pamphlet" files. Lets say you save it as +"foo.pamphlet". You now need 2 tools, one to extract the +.tex file (noweave) and one to extract the .lisp file (notangle) thus: + +noweave foo.pamphlet >foo.tex +notangle foo.pamphlet >foo.lisp (or .c or .boot, etc) + +Thus the pamphlet file contains both the code and the documentation. +The idea is to take the boot code and your thesis, mix the two, +write the intermediate explanations that go from the theory to +the code chunks, and create a pamphlet file (or many pamphlet files) +that document what the code does and why. + +You'll be unique as everyone eventually asks how axiom does +coercion and there has never been a good answer to that question +before. + +I'm willing to do all of the dog work on making the pamphlet files +if you'll send me the code and the tex files. Once they are mixed +I can build it into the build process and we can work together on +improving the form and content of the pamphlets. + +\start +Date: Thu, 11 Dec 2003 21:48:56 +0100 +From: David MENTRE +To: Bertfried.Fauser@uni-konstanz.de +Subject: Re: [Axiom-developer] Re: Axiom interactive input syntax +Bertfried Fauser writes: + +> browse Ring +> +> Ring +> |___ Euclidean Ring +> | |____ Division Ring +> |___ Artinear Ring +> |... +> ... +> +> which shows which types are subtypes, (downsearch) +> +> A second way of working should diplay the upbraces of teh type tree +> (graph?) so that you see all datastructures which contain the type +> Ring, that may be modules, groups, abelian monoids, etc.... + +Thank you Bertfried, it's more clearer now. + +\start +Date: Thu, 11 Dec 2003 22:15:05 +0100 +From: David MENTRE +To: Tim Daly +Subject: [Axiom-developer] GNU Arch (was: Re: [Axiom-math] Re: phd thesis) + +Hello Tim, + +Tim Daly writes: + +> I expect there will be several "research" branches because I know of +> others who want to experiment with axiom. Indeed, axiom's basic +> strength is that it is a very good platform for research. + +By the way and for information, GNU Arch is a version control system +(cvs-like) specifically designed to allow separate trees to merge in a +bazaar style. In another words, Arch would match the needs for several +individual experimental research Axioms. I'm not advocating it since +I've never used it. But a TeXmacs developer recently made its promotion +on texmacs-dev mailing list. + +\start +Date: Thu, 11 Dec 2003 19:25:26 -0500 +From: "Page, Bill" +To: nic@uklinux.net +Subject: [Axiom-developer] RE: phd thesis + +On Thursday, December 11, 2003 12:31 PM Tim Daly +[mailto:daly@rio.sci.ccny.cuny.edu] wrote: +> ... +> I've uploaded a very late (possibly not the latest) version of +> Axiom to my own website. You can get it and build it by: +> +> cd (yourpath) +> visit axiom.tenken.org +> download the "20031211 sources" +> tar -zxf axiom.20031211.tgz +> cd axiom +> export AXIOM=(yourpath)/axiom/mnt/linux +> make +> + +That's actually (just a typo): + + http://axiom.tenkan.org/ + +> re: solaris, etc +> +> >I also have access to Red Hat Enterprise Linux 2.1 and 3 +> >boxen as well as Solaris 8 if there's any porting help +> >needed. I can also build RPMs... +> +> The other boxen are interesting, particularly the solaris 8 +> version. I've tried to download solaris86 to do a build but +> that whole process failed and I'll have to try again. If you +> want to try a port you should only need to modify the top level +> makefile but you'll almost certainly need my help with that. +> + +Tim, I have an accessible Solaris 8 (on sparc) machine. I will +also try a build when I get a few spare minutes. + +\start +Date: Thu, 11 Dec 2003 20:09:20 -0500 +From: "Page, Bill" +To: "'Tim Daly'" +Subject: [Axiom-developer] Access to Savannah (was RE: phd thesis) + +On Thursday, December 11, 2003 12:31 PM Tim Daly +[mailto:daly@rio.sci.ccny.cuny.edu] wrote: + +> ... +> re: fedora core +> +> The build will almost work fine on Fedora (I'm also a fedora developer +> and have been campaigning against the change that breaks axiom). In +> order to get around the problem you need to become root and type: +> +> echo 0 >/proc/sys/kernel/exec-shield +> +> Once that is done you can correctly build Axiom as a regular user. +> In theory you can download Axiom by incanting: +> +> cd (yourpath) +> cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom login +> cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom co axiom +> cd axiom +> export AXIOM=(yourpath)/axiom/mnt/linux +> make +> +> When prompted for a password just hit enter. +> +> However, the gnu website is down due to an audit. +> + +The (unaudited) Savannah cvs files are now available at + + http://savannah.gnu.org/statement.html + +where you will find the following instructions: + +--------- + +For example, to check out the CVS module "axiom" in the Savannah +project of the same name, as it existed in the Savannah CVS tree +when we brought the system down, you would need to use the following +commands: + + export CVS_RSH="ssh" + cvs -d:ext:anoncvs@subversions.gnu.org:/cvs-latest/axiom co axiom + +If you already have a subversions.gnu.org entry in your ~/.ssh/config +file with a "Protocol 1" line, you will need to change it to "Protocol +2". + +--------- + +I just tested this and it works for me. + +\start +Date: Fri, 12 Dec 2003 17:07:12 +0300 +From: "Vadim V. Zhytnikov" +To: Camm Maguire +Subject: [Axiom-developer] Re: [Gcl-devel] GCL on mingw +Cc: Mike Thomas + +Camm Maguire ?????: + +> Greetings! +> +> "Vadim V. Zhytnikov" writes: +> +> +>>Camm Maguire ?????: +>> +>> +>>>Hi Vadim! +>>>"Vadim V. Zhytnikov" writes: +>>> +>>> +>>>>Hi! +>>>> +>>>>Trying to build recent GCL 2.6.1 on mingw I encountered +>>> +>>> ^^^^^^^^^^^^^^^ +>>>Great! I was just going to ask you if you had a mingw development +>>>system to work with given your earlier mingw problem report. +>>> +>> +>>Well, I just followed Mike's readme.mingw instruction. +>>The key point is additional (si::use-fast-links nil) in +> +> +> ^^^^^^^^^^^^^^^^^^^^^^^ +> +> Thanks for the reminder. I had forgotten about this. I'm wondering +> if this is the only place where fast-links cause a problem. If so, it +> may simply be in the large array GCL allocates for the purposes of +> toggling between fast and slow function pointers coupled with the +> memory allocation issues you are observing.. If not, perhaps there is +> some alignment issue, or (much worse), some ia64-like retrictions on +> function calls via function pointers when the shared library maps +> could change across runs. We should be able to break at the bad +> function jump in pcl_braid.c (see earlier thread with the debugging +> details if interested), go up one frame, disassemble, and look at the +> registers to see if the function address has somehow been corrupted. +> +> +>>pcl/makefile. He also apparently recommends --enable-custreloc +>>but I was able to build ANSI GCL with and without this option. +> +> +> ??? Meaning via statsysbfd (the default), or via dlopen? +> +> +>>On the other hand I did it with some gcl 2.6.1 snapshot not +>>with very last CVS sources - I have to try it once again. +>> +> +> +> It would be nice if you could also verify the ansi build crash when +> not turning off fast-links and building without custreloc. +> + +I've build latest ANSI GCL (Dec 03, 2003) without custreloc +(all default options except --enable-ansi). CONS allocation test +fails at the very beginning of 8th pass during RB GBC with +the message + Can't allocate. Good-bye! +This is a bit different from traditional GCL. I think that +this difference is purely due to different initial +memory layouts of traditional and ansi images. + +\start +Date: 12 Dec 2003 10:59:22 -0500 +From: Camm Maguire +To: "Vadim V. Zhytnikov" +Subject: [Axiom-developer] Re: [Gcl-devel] GCL on mingw +Cc: Mike Thomas + +Greetings! + +"Vadim V. Zhytnikov" writes: + +> Camm Maguire ?????: +> +> > Greetings! +> > "Vadim V. Zhytnikov" writes: +> > +> >>Camm Maguire ?????: +> >> +> >> +> >>>Hi Vadim! +> >>>"Vadim V. Zhytnikov" writes: +> >>> +> >>> +> >>>>Hi! +> >>>> +> >>>>Trying to build recent GCL 2.6.1 on mingw I encountered +> >>> +> >>> ^^^^^^^^^^^^^^^ +> >>>Great! I was just going to ask you if you had a mingw development +> >>>system to work with given your earlier mingw problem report. +> >>> +> >> +> >>Well, I just followed Mike's readme.mingw instruction. +> >>The key point is additional (si::use-fast-links nil) in +> > ^^^^^^^^^^^^^^^^^^^^^^^ +> > Thanks for the reminder. I had forgotten about this. I'm wondering +> > if this is the only place where fast-links cause a problem. If so, it +> > may simply be in the large array GCL allocates for the purposes of +> > toggling between fast and slow function pointers coupled with the +> > memory allocation issues you are observing.. If not, perhaps there is +> > some alignment issue, or (much worse), some ia64-like retrictions on +> > function calls via function pointers when the shared library maps +> > could change across runs. We should be able to break at the bad +> > function jump in pcl_braid.c (see earlier thread with the debugging +> > details if interested), go up one frame, disassemble, and look at the +> > registers to see if the function address has somehow been corrupted. +> > +> >>pcl/makefile. He also apparently recommends --enable-custreloc +> >>but I was able to build ANSI GCL with and without this option. +> > ??? Meaning via statsysbfd (the default), or via dlopen? +> > +> >>On the other hand I did it with some gcl 2.6.1 snapshot not +> >>with very last CVS sources - I have to try it once again. +> >> +> > It would be nice if you could also verify the ansi build crash when +> > not turning off fast-links and building without custreloc. +> > +> +> I've build latest ANSI GCL (Dec 03, 2003) without custreloc +> (all default options except --enable-ansi). CONS allocation test + +Just checked, and the configure default for mingw is cust-reloc. I'm +imagining that --disable-custreloc --enable-locbfd will fail. + +> fails at the very beginning of 8th pass during RB GBC with +> the message +> Can't allocate. Good-bye! + +Sorry for the confusing request. I wanted to see if you can reproduce +the crash when building pcl from source as part of the ansi build +process when *not* turning off fast-links. I'm inferring that +cust-reloc must be used on mingw, so I imagine that you will easily +reproduce Mike's reported problem. Still, it would be helpful to +reproduce this in a directory with --enable-debug turned on so we can +see what the problem is (i.e. whether it is memory related or not.) + +> This is a bit different from traditional GCL. I think that +> this difference is purely due to different initial +> memory layouts of traditional and ansi images. + +\start +Date: Fri, 12 Dec 2003 18:09:50 +0000 +From: nic +To: Tim Daly +Subject: [Axiom-developer] Re: phd thesis +Cc: bill.page1@sympatico.ca, Bertfried.Fauser@uni-konstanz.de + +On Thu, 2003-12-11 at 17:30, Tim Daly wrote: + +I've located all my source code. I've now got to understand it and work +out which was the more recent version. ;-) + +\start +Date: Mon, 15 Dec 2003 11:53:56 +0000 +From: nic +To: David MENTRE +Subject: Re: Type coercicion (was: Re: [Axiom-developer] Re: Axiom interactive input syntax) + +On Wed, 2003-12-10 at 21:19, David MENTRE wrote: +> Hello, +> +> nic writes: +> +> > I did some work on automatic type changing in the interpreter for my +> > thesis under James Davenport... +> +> >From what I have understood from the introduction and conclusion, you +> have an algorithm to transform any "type"[1] in any other mathematically +> compatible "type" (provided the type system follows some mathematical +> guidelines). This makes me ask the following na=EFve and meta question: +> how would you use such an algorithm? From user behavior (tim-user uses +> variable x with another operator implying a new type context) or with +> explicit user information (calls to coerce)? Within a type inference +> algorithm? + +It "should" be invisible to the user (ie. during an internal call to +coerceInteractive() ). + +The standard case being that Q[x] has a natural injection into Z(x) +( UPoly Frac Int -> Frac UPoly Int ). + +So if a :=3D ( ( 1/2 ) * x**2 + x - 1/3 ) (Upoly Frac Int) we are free to +add a member of Z, Q, Z(x) to it and will "work" without the user having +to call coerce. + +That's not to say that it shouldn't be callable by the user, somehow. + +It should also be noted that I think my algorithm is slow. The proof in +the thesis is a reverse-engineered hack. :-) Finally, I don't think +we'll be in a position to have Axiom's language like B-natural +(described in ISSAC '94: "How to Make AXIOM into a Scratchpad" Jenks and +Trager). There will always be cases where users have to coerce (or +worse, cast e.g. Q -> Float). I don't think that my work is a panacea +for all ills but it should help and is mathematically sound in a large +number of day-to-day cases. + +\start +Date: Thu, 18 Dec 2003 02:49:45 -0500 +From: root +To: savannah-compromise@gnu.org +Subject: [Axiom-developer] audit + +Gentlepersons, + +I have completed a review of the axiom-changes.tar.gz file as posted +on your website (http://savannah.gnu.org/statement.html). All of the +changes appear to be valid. Therefore Axiom can be considered as +"clean" as of the time you did the diff. + +\start +Date: Tue, 23 Dec 2003 15:57:09 +0100 +From: David MENTRE +To: axiom-mail@nongnu.org, axiom-developer@nongnu.org +Subject: [Axiom-developer] Savannah is back + +Hello Axiom users and developers, + +I don't know if Axiom users ou developers have been annoyed but +Savannah, the infrastructure provided by the Free Software Foundation +(FSF) is now on-line after several weeks of down-time due to a system +crack. + +We have now access to CVS trees and others development tools but several +parts are still missing, like the download area or the update of the Web +pages[1]. + +Tim Daly, Axiom project leader, has checked the Axiom sources and +confirmed that Axiom sources has NOT been compromised[2]. + +The Axiom project members will have to get used to new procedures and +will have to find new temporary solutions to lack of download area. So +expect a little more issues for the Axiom project is the short term. + +Beside that, for the whole Axiom team, I wish you a nice christmas. + +[1] http://savannah.nongnu.org/forum/forum.php?forum_id=2752 +[2] http://mail.gnu.org/archive/html/axiom-developer/2003-12/msg00052.html + +\start +Date: Tue, 23 Dec 2003 10:24:43 -0500 +From: Tim Daly +To: david.mentre@wanadoo.fr +Subject: [Axiom-developer] re: Axiom is back +Cc: axiom-developer@nongnu.org, daly@idsi.net, axiom-mail@nongnu.org + +*, + +I got email this morning from Brad Kuhn at FSF about Savannah. +He details new procedures for fetching and updating Axiom. +I am going to try these procedures (as well as a few project-lead +tasks I need to do) and then I'll send a note to these mailing +lists to clarify how you can again access Axiom. + +The new procedures are mildly more painful but they are supposed to +prevent this kind of distruption in the future. + +Happy holidays to all. + +\start +Date: 23 Dec 2003 17:27:10 +0100 +From: WZocher@t-online.de (Wolfgang Zocher) +To: axiom-developer +Subject: [Axiom-developer] Merry Christmas + +A Merry Christmas to all developers, users and friends of Axiom from + +-- +Wolfgang Zocher http://wzocher.bei.t-online.de/ +Registered Linux User #337888 using Debian GNU/Linux + +\start +Date: Sun, 28 Dec 2003 15:06:06 -0500 +From: root +To: bill.page1@sympatico.ca +Subject: [Axiom-developer] documentation and the crystal +Cc: gilbert@sci.ccny.cuny.edu + +Bill, + +per our long-lost prior discussion i've been documenting axiom. +the idea of documenting a system like axiom has several facets +as i've come to discover. what it means to document axiom has +a lot to do with why you are looking at the documentation. so +in some way the documentation has to be structured into something +like a large crystal egg with many facets. somehow we've got to +design a documentation structure that will allow the same basic +information to be used in dozens of different ways depending on +why you're looking at it. consider that axiom needs documenting: + +of the internal structure of the system + the data structures + the logical layers (c, lisp, boot, spad) + the functional separation (compiler, interpreter, browser) + the system-dependent (sockets) vs independent (lisp) +of the external structure + the interpreter + the compiler + the graphics + the hyperdoc + the openmath + the documentation +of the algebra structure + the packages + the domains + the categories +of the mathematical structure + the subjects covered + the theory underlying the subjects + categories +of the computational mathematics structure + intermediate expression swell + simplification + math types vs computational types +of the user structure (the book) + the commands for naviation + the commands for documentation + the available math functions +of the programming language + the compiler syntax + the compiler semantics + sequencing, conditionals, looping, file i/o + domain construction + categorical tests/constraints +of the testing structure + the mathematics underlying the tests (CATS, computer algebra test suite) + the actual tests + boundary conditions +of the literature + published algorithms + published theory + thesis work +of the program proofs + underlying theorems and lemmas + program proof books + +i'm currently mucking around in the algebra structure. in particular +i've catted all of the algebra together into one file and am "reducing" +it to it's primitive structure. this alone is a daunting task as it +starts out with about a quarter million lines which i've slowly reduced +to about 100k lines so far. i'm doing a topological sort of the algebra +to uncover the actual type hierarchy with the idea that it can be +reduced to a lattice. as you recall this problem was done once before +in order to get axiom to compile from scratch. + +getting down to this level of detail for documenting makes it clear +that current systems of documenting are hopelessly weak. somehow we +need to take advantage of the computer to leverage and reuse documentation +in creative ways. if we don't we'll just drown in endless documents. ibm +was famous for delivering shelves worth of documentation which was never +used. barnes and noble has whole bookcases of documents on linux. that +way lies madness. + +in fact, documentation is probably the wrong idea. we need somehow to +be able to automatically generate information from some core that +represents the axiom system itself. + +so i'm thinking about a "crystal browser", that is, a browser where you +can gaze into a crystal that surrounds axiom. each facet represents a +generated view of the whole system. trivially we could have a facet +that shows algebra source code. we could also have a facet that shows +the type hierarchy, etc. so it is clear we can create automatic facets +with just the existing code and programs that do structure analysis. + +more generally we could construct facets that walk into the pamphlets. +one facet could walk the biblio references, another could extract the +tex, a third could walk index terms to find all references to a single +term (e.g. ideal). particularly interesting would be facets that walk +the semantic structure of the system so you could pick out particular +kinds of ideals or proofs using ideals, etc. certain facets could be +used to impose order by certain metrics (like rainbows in real +crystals). such rainbow facets could show the type lattice ordered +by layer (ala the structure in the src/algebra/makefile). yet more +generally is that "literate programs" need to have sufficient structure +to support the crystal. + +in particular, we need to look at some technology that will do some +automated work for us. one that leaps to mind is a semantic network. +new code would be automatically classified into the system based on +several factors (some hand supplied, some derived from the code). + +the idea that it is "just documentation" and "just a browser" is +a weak notion but a good start. in general one would like to use +facets to CONSTRUCT new algebra, new booklets, new proofs, etc. +so both "documentation" and "browser" are the wrong words. + +in 30 year computational mathematicians will need to be able to +deal with the complexity of all of the facets of documentation +i mentioned above. we need to construct tools which help organize +the system in ways that a mathematician can effectively use to do +research work. + +the general "visual image" is of a large crystal which you can rotate. +every facet gives a different view of the whole of the axiom system. +thus, a "crystal" surrounding axiom. + +hope your christmas went well. + +\start +Date: Sun, 28 Dec 2003 15:23:58 -0500 +From: root +To: bill.page1@sympatico.ca +Subject: [Axiom-developer] documentation and the crystal +Cc: gilbert@sci.ccny.cuny.edu + +Bill, + +more top level speculation about the crystal leads to an interesting +problem. right now we have the image of books and standard page +navigation allows us to put "next", "previous", "index", etc on +each page as navigation buttons. + +at the other extreme is the web where "forward" and "back" are all +of the possible navigation choices. + +with a crystal one assumes that each facet is somehow "near" to the +adjacent facets. since this is a virtual object the facets don't need +to be all equally shaped (e.g. 6 sided). some facets could have many +possible neighbors (where information views are rich) and some could +have relatively few neighbors. the notion of "next" facet could be +specified by an angle measure of some sort. "forward" and "back" +become paths on the crystal surface. hyperlinking can move from any +facet to any other. in particular, viewing a "corner" could show +several facets at once (e.g. code, it's location in the lattice, +and the documentation for that code) which are all joined at a +virtual corner. + +more generally one could embed one crystal within another. so the +inner crystal facet shows the type hierarchy and the outer crystal +facet looks at code for a particular item in the hierarchy. this +is the 3-D notion of 1-D pipes, that is, information from one source +is filtered thru a pipe and shown thru another source. (e.g. grep +the algebra for "Category" and filter out only the lines that define +the domain). + +the crystal is a virtual object which is really nothing more than +a way to organize hundreds of functions to manipulate code. the +crystal could be specified as a file containing its structure +definition where each facet is just a series of pipe-connected +functions and a list of adjacent facets with navigation tags. +it could be built on top of a standard browser. + +if done right the crystal should not be axiom specific, only the +functions need to know the details of their data sources. + +\start +Date: Sun, 28 Dec 2003 22:06:41 -0500 +From: "Bill Page" +To: +Subject: RE: [Axiom-developer] documentation and the crystal +Cc: gilbert@sci.ccny.cuny.edu + +Tim, + +Well, I have often used the time between christmas and +new years for thinking what I hope are radical and +forward looking thoughts. It sounds to me like you share +this tradition! + +We had a pleasant christmas but work stress and "holiday +stress" combined to give Faye a serious case of pneumonia. +She has been confined to bed for the last three days (but +improving) so that has significantly dampened our enjoyment +of the extraordinarily mild weather ... I trust that in +your case things went a little better. :) + +I have also been thinking about Axiom documentation. I +definitely agree that the book (or even the "IBM library") +metaphor is not adequate for a system as complex as Axiom. +I realize that you are probably writing in an online +"brainstorming" style, but have to admit that your "crystal" +metaphor also tends to leave me feeling a little cold. +However, when you said "semantic network" this connected! + +There is a large and growing literature on the "semantic +web". + + http://www.w3.org/2001/sw/ + +"Definition: The Semantic Web is the representation of +data on the World Wide Web. It is a collaborative effort +led by W3C with participation from a large number of +researchers and industrial partners. It is based on the +Resource Description Framework (RDF), which integrates a +variety of applications using XML for syntax and URIs for +naming." + +In fact, NAG (Mike Dewar) has been one of the leading +organizations promoting the concept of mathematics in +the context of the semantic web. + + = +http://monet.nag.co.uk/cocoon/openmath/meetings/eindhoven2003/index.html + +See especially Mike's presentation on Monet + +http://monet.nag.co.uk/cocoon/openmath/meetings/eindhoven2003/proceedings= +/de +war-monet.htm + +---------- + +So rather than "crystal", I am inclined to think of +the Axiom documentation (and programs) as a "web". In +fact, the Axiom web could fundamentally reside on a +web application server such as + + http://www.zope.org + +Zope provides a high level object-oriented environment +using tools that are not so different than Axiom itself +(Python). + + http://www.zope.org/WhatIsZope + +Using these tools together with some of the standards +discussed by Mike Dewar, I could imagine configuring Axiom +(and in the longer term, the network of Axiom developers) +into a dynamic distributed active environment for +mathematics ... [Ok, that's as far as I go with the hype.] + +But seriously, as strange as it might sound at first, +what more suits the multidimensional network structure +of Axiom itself better than the "web". And the further we +look into your "thirty year" time horizon, the more sense +this makes to me. + + +> -----Original Message----- +> From: +> axiom-developer-bounces+bill.page1=3Dsympatico.ca@nongnu.org +> [mailto:axiom-developer-bounces+bill.page1=3Dsympatico.ca@nongnu +> .org] On Behalf Of root +> Sent: Sunday, December 28, 2003 3:06 PM +> To: bill.page1@sympatico.ca +> Cc: gilbert@sci.ccny.cuny.edu; axiom-developer@nongnu.org; +> axiom-math@nongnu.org; daly@idsi.net +> Subject: [Axiom-developer] documentation and the crystal +> +> +> Bill, +> +> per our long-lost prior discussion i've been documenting +> axiom. the idea of documenting a system like axiom has +> several facets as i've come to discover. what it means to +> document axiom has a lot to do with why you are looking at +> the documentation. so in some way the documentation has to be +> structured into something like a large crystal egg with many +> facets. somehow we've got to design a documentation structure +> that will allow the same basic information to be used in +> dozens of different ways depending on why you're looking at +> it. consider that axiom needs documenting: +> +> of the internal structure of the system +> the data structures +> the logical layers (c, lisp, boot, spad) +> the functional separation (compiler, interpreter, browser) +> the system-dependent (sockets) vs independent (lisp) +> of the external structure +> the interpreter +> the compiler +> the graphics +> the hyperdoc +> the openmath +> the documentation +> of the algebra structure +> the packages +> the domains +> the categories +> of the mathematical structure +> the subjects covered +> the theory underlying the subjects +> categories +> of the computational mathematics structure +> intermediate expression swell +> simplification +> math types vs computational types +> of the user structure (the book) +> the commands for naviation +> the commands for documentation +> the available math functions +> of the programming language +> the compiler syntax +> the compiler semantics +> sequencing, conditionals, looping, file i/o +> domain construction +> categorical tests/constraints +> of the testing structure +> the mathematics underlying the tests (CATS, computer +> algebra test suite) +> the actual tests +> boundary conditions +> of the literature +> published algorithms +> published theory +> thesis work +> of the program proofs +> underlying theorems and lemmas +> program proof books +> +> i'm currently mucking around in the algebra structure. in +> particular i've catted all of the algebra together into one +> file and am "reducing" it to it's primitive structure. this +> alone is a daunting task as it starts out with about a +> quarter million lines which i've slowly reduced to about 100k +> lines so far. i'm doing a topological sort of the algebra to +> uncover the actual type hierarchy with the idea that it can +> be reduced to a lattice. as you recall this problem was done +> once before in order to get axiom to compile from scratch. +> +> getting down to this level of detail for documenting makes it +> clear that current systems of documenting are hopelessly +> weak. somehow we need to take advantage of the computer to +> leverage and reuse documentation in creative ways. if we +> don't we'll just drown in endless documents. ibm was famous +> for delivering shelves worth of documentation which was never +> used. barnes and noble has whole bookcases of documents on +> linux. that way lies madness. +> +> in fact, documentation is probably the wrong idea. we need +> somehow to be able to automatically generate information from +> some core that represents the axiom system itself. +> +> so i'm thinking about a "crystal browser", that is, a browser +> where you can gaze into a crystal that surrounds axiom. each +> facet represents a generated view of the whole system. +> trivially we could have a facet that shows algebra source +> code. we could also have a facet that shows the type +> hierarchy, etc. so it is clear we can create automatic facets +> with just the existing code and programs that do structure analysis. +> +> more generally we could construct facets that walk into the +> pamphlets. one facet could walk the biblio references, +> another could extract the tex, a third could walk index terms +> to find all references to a single term (e.g. ideal). +> particularly interesting would be facets that walk the +> semantic structure of the system so you could pick out +> particular kinds of ideals or proofs using ideals, etc. +> certain facets could be used to impose order by certain +> metrics (like rainbows in real +> crystals). such rainbow facets could show the type lattice +> ordered by layer (ala the structure in the +> src/algebra/makefile). yet more +> generally is that "literate programs" need to have sufficient +> structure +> to support the crystal. +> +> in particular, we need to look at some technology that will +> do some automated work for us. one that leaps to mind is a +> semantic network. new code would be automatically classified +> into the system based on several factors (some hand supplied, +> some derived from the code). +> +> the idea that it is "just documentation" and "just a browser" is +> a weak notion but a good start. in general one would like to +> use facets to CONSTRUCT new algebra, new booklets, new +> proofs, etc. so both "documentation" and "browser" are the +> wrong words. +> +> in 30 year computational mathematicians will need to be able +> to deal with the complexity of all of the facets of +> documentation i mentioned above. we need to construct tools +> which help organize the system in ways that a mathematician +> can effectively use to do research work. +> +> the general "visual image" is of a large crystal which you +> can rotate. every facet gives a different view of the whole +> of the axiom system. thus, a "crystal" surrounding axiom. +> +> hope your christmas went well. + +\start +Date: Mon, 29 Dec 2003 15:14:36 +0100 +From: David MENTRE +To: daly@idsi.net +Subject: [Axiom-developer] Re: [Axiom-math] documentation and the crystal +Cc: gilbert@sci.ccny.cuny.edu + +Hello Tim and Bill, + +You are right, holidays are always a good period to dig into new ideas. + +I think I have the same approach as both of you, Tim and Bill. Regarding +the Semantic Web, Bill, I think it is a interesting technology that we +might look at and use as a reference. But for me, it is just a +technology. We need to refine first our own ideas on what we want to not +lose our 30 years goal. That's said, you are right that smart people +have invested time in this technology and it would be stupid of us to +not resuse the wheel. + +Tim, regarding your crystal approach, I particularly like it, at least +for the several "facets" to look through one complex system. However, I +would prefer to use a graph-like approach, which in my opinion is +probably more related to the Knowledge Network you have spoken about. + +Usually, people have a top-down approach: they modelize a system +abstractly (using more or less formal notations as UML or SDL) and then +refine them through the actual code. But, in the case of Axiom, we need +the reverse. We need to start from concrete objects (files, lines of +source code) and add semantics to climb levels of abstraction. Of +course, you follow different ladders, in the sense that understanding +the compiler or the algebra would need different information and is +structured differently, thus the different crystal facets of Tim. + + +More concretely, I would propose the following approach: + + 1. start from parsers for the src/ directory. Parse directory structure + and each file, categorize them (boot, lisp, spad, ...) and construct + basic abstractions (list of lisp and boot functions; list of spad + categories, domains and operators; ...) + + 2. from information extracted in step 1, construct one or several + representation (knowledge graph for example) with the found + semantics (name and body of a function for example), probably using + a standard technology as W3C semantic web + + 3. the "Axiom explorer" (Tim, Bill or I) is interested in a specific + "crystal facet" (e.g. the compiler). He builds another tool + (e.g. call-graph analyzer for lisp code) which in turn is used to + construct a new knowledge graph, concretized in additional + information in the W3C semantic web. + + 4. Within the abstraction level build in step 4 (or 2), the "Axiom + explorer" adds its own knowldege (e.g. this set of functions is used + to parse Spad code, this other set is used for type analysis, ...) + to the semantic web. + + 5. using previous abstraction levels and probably building a new tool, + the "Axiom explorer" iterates, climbing abstraction levels, until he + reaches his own goal. + +Repeat steps 1 to 5 with enough people to cover Axiom from A (Algebra) +to Z (zerodim). + +Ok. I might be a bit optimistic and what I have said might appear more +than abstract, but this is the current state of might thoughts. :) + +Even if you do not like above ideas, I think following "principles" are +needed for a documentation system for Axiom (principles already +formulated by Tim is his first email): + + o separation principle: in engineering, people separate a complex issue + in _independant_ sub-issues to be able to understand them and solve + them independently. We probably need to deconstruct Axiom in + independent (or a least loosely connected) sub-systems to be able to + understand them (i.e. the different crystal facets of Tim); + + o "build on giant shoulders" principle: we need a way to reuse + knowledge from other "crystal facets". For example, I would use Tim + knowledge of the internals of Axiom to understand how the compiler + compiles a given portion of the algebra; + + o automation principle: Axiom is too big to add information manually on + each function, each object, etc. We need tools to annotate a set of + objects given a selection criteria (e.g. all operators in this Spad + domain). + +New year wish: I'll try to write and "show you the code" for above +ideas. :) My own todo for this subproject of Axiom would be: + + - learn more about W3C Semantic Web (thank you Bill for the pointer) + + - find or write tools to manipulate Semantic Web (it might be Emacs + with a proper mode or a more elaborated graphical tool) + + - apply above approach, starting from directory structure in src/ + directory of Axiom. + + - from this first experiment, think about what would need to be + "standardized", like common dictionnary or vocabulary, etc. Beyond + usual technological issues, I think this point is one fo the harder + point. How to build a set of knowledge that will be still useful in + 30 years from now? + + +By the way, does anybody knows about a library to *draw* and manipulate +arbitary graphs (in Common Lisp, C++, ML, ...) in a user interface? I +know about DOT or VCG but they dot not match my needs: I would like to +draw a graph, know when the user click on a graph node or edge and react +accordingly. Does anybody know where I could find such an already made +tool? Any knowledge of a browser for the W3C semantic web? + +\start +Date: 29 Dec 2003 16:07:32 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: [Axiom-developer] si::sgc-on t in Axiom? + +Greetings! I'm just wondering if anyone has had experience building +axiom with the 'sgc' feature in gcl turned on. This is a write memory +barrier intended to speed up garbage collection. My initial attempts +are leading axiom (when compiling OUT.NRLIB) to complain with + +$ ../.././obj/linux/bin/interpsys +GCL (GNU Common Lisp) (2.6.1) Thu Nov 6 16:23:13 UTC 2003 +Licensed under GNU Library General Public License +Dedicated to the memory of W. Schelter + +Use (help) to get some basic information on how to use GCL. + +(AXIOM Sockets) The AXIOM server number is undefined. +----------------------------------------------------------------------------- + Issue )copyright to view copyright notices. + Issue )summary for a summary of useful system commands. + Issue )quit to leave AXIOM and return to shell. +Monday December 29, 2003 at 19:53:49 +----------------------------------------------------------------------------- + + Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/compress.daase.. Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/interp.daase.. + Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/operation.daase.. + Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/category.daase.. + Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/browse.daase.. +(1) -> )co OUT.spad + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/apply. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/c-doc. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/c-util. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/profile. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/category. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/compiler. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/define. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/functor. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/info. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/iterator. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/modemap. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/nruncomp. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/package. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/htcheck. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/xruncomp. + Compiling AXIOM source code from file + /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/OUT.spad using old + system compiler. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/parsing. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/bootlex. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/def. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/fnewmeta. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/metalex. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/metameta. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/parse. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/postpar. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/postprop. + Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/preparse. + OUT abbreviates package OutputPackage +------------------------------------------------------------------------ + initializing NRLIB OUT for OutputPackage + compiling into NRLIB OUT + processing macro definition E ==> OutputForm + processing macro definition putout ==> elt(Lisp,mathprint) + compiling exported output : OutputForm -> Void +Time: 0.01 SEC. + + compiling exported output : String -> Void +Time: 0 SEC. + + compiling exported output : (String,OutputForm) -> Void +Time: 0.03 SEC. + + compiling exported outputList : List Any -> Void + Internal Error + Error while instantiating type Any + +protected-symbol-warn called with (NIL) + + +I obviously haven't done much research yet -- just asking if anyone +else has experience. + +\start +Date: 29 Dec 2003 17:04:13 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: Re: [Axiom-developer] si::sgc-on t in Axiom? + +Greetings! + +Please disregard this message -- it appears to be working with the +latest SGC fix I've committed for some UT people. This fix is already +in 2.6.1-19 which is about to (in a few days) become the official +2.6.2 release. I'll post comparisons if/when I get a chance. + +Take care, + +Camm Maguire writes: + +> Greetings! I'm just wondering if anyone has had experience building +> axiom with the 'sgc' feature in gcl turned on. This is a write memory +> barrier intended to speed up garbage collection. My initial attempts +> are leading axiom (when compiling OUT.NRLIB) to complain with +> +> $ ../.././obj/linux/bin/interpsys +> GCL (GNU Common Lisp) (2.6.1) Thu Nov 6 16:23:13 UTC 2003 +> Licensed under GNU Library General Public License +> Dedicated to the memory of W. Schelter +> +> Use (help) to get some basic information on how to use GCL. +> +> (AXIOM Sockets) The AXIOM server number is undefined. +> ----------------------------------------------------------------------------- +> Issue )copyright to view copyright notices. +> Issue )summary for a summary of useful system commands. +> Issue )quit to leave AXIOM and return to shell. +> Monday December 29, 2003 at 19:53:49 +> ----------------------------------------------------------------------------- +> +> Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/compress.daase.. Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/interp.daase.. +> Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/operation.daase.. +> Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/category.daase.. +> Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/browse.daase.. +> (1) -> )co OUT.spad +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/apply. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/c-doc. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/c-util. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/profile. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/category. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/compiler. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/define. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/functor. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/info. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/iterator. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/modemap. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/nruncomp. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/package. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/htcheck. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/xruncomp. +> Compiling AXIOM source code from file +> /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/OUT.spad using old +> system compiler. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/parsing. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/bootlex. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/def. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/fnewmeta. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/metalex. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/metameta. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/parse. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/postpar. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/postprop. +> Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/preparse. +> OUT abbreviates package OutputPackage +> ------------------------------------------------------------------------ +> initializing NRLIB OUT for OutputPackage +> compiling into NRLIB OUT +> processing macro definition E ==> OutputForm +> processing macro definition putout ==> elt(Lisp,mathprint) +> compiling exported output : OutputForm -> Void +> Time: 0.01 SEC. +> +> compiling exported output : String -> Void +> Time: 0 SEC. +> +> compiling exported output : (String,OutputForm) -> Void +> Time: 0.03 SEC. +> +> compiling exported outputList : List Any -> Void +> Internal Error +> Error while instantiating type Any +> +> protected-symbol-warn called with (NIL) +> +> +> I obviously haven't done much research yet -- just asking if anyone +> else has experience. + +\start +Date: Tue, 30 Dec 2003 11:24:50 -0500 +From: root +To: bill.page1@sympatico.ca, alexo@newmail.ru, gilbert@sci.ccny.cuny.edu, david.mentre@wanadoo.fr +Subject: [Axiom-developer] crystal and the semantic web + +*, + +Sorry for the delay in reply. My inbound mail queue got silently hosed. +Working on it now. + +\start +Date: Tue, 30 Dec 2003 10:38:48 -0500 +From: "Bill Page" +To: +Subject: RE: [Axiom-developer] crystal and the semantic web +Cc: gilbert@sci.ccny.cuny.edu, alexo@newmail.ru + +On Tuesday, December 30, 2003 11:25 AM Tim wrote: +> +> Sorry for the delay in reply. My inbound mail queue got +> silently hosed. Working on it now. +> + +I guess that it is the season to get "hosed", silently, +in the night ... + +\start +Date: Tue, 30 Dec 2003 19:07:15 +0100 (CET) +From: Bertfried Fauser +To: root +Subject: Re: [Axiom-developer] documentation and the crystal +Cc: gilbert@sci.ccny.cuny.edu, Bill Page + +Dear Christmas holiday workers! + +beside wishing all the best for the new year, I would like to ask a few +more questions about axiom documentation. + +a) To me its not even clear how to structurize mathematics. One cxan +build mathematics on sets, hence the Bourbaki approch, or even better (in +my eyes, but equivalent in formal strength) on the *function first* +principle. What to choose for axiom? (In fact the set approch is build-in) + +b) I do not see how you can automatically assign semantics to data +strutures etc. I think, one has at least to have a sort of *semantic +typing* during the documentation of axiom (code). Hence every piece of +documentation should come with a semantic type (or multiple such types) +which finally allow to put a direction into the crystal looking glass. + +c) Algorithms should be plain and readable and not only be available in +code form (if even this way) + +d) To me it would be much more natural to look at the documentation like a +big database and ask SQL like questions. EG: + +> select from AXIOM algorithm where domain has commutative; + +or such. And then get a sort of (semantic? web?) document which allows to +go deeper into an algorithm, eg, thet should be lionks to al faces of the +crystal which make sense to look at. + +e) The system will not be more smart than its designers / users. I do not +see how an automated method will derive anything beyong mere syntactical +sorting. To be frak I have no idea how to reach the above needs. + +f) As a probably managable project, it would be of utmost importance for +me (and other mathematically interested, who are stupid programers) to +have just the functionality described above for teh algebra lattice. I had +such an email with David Mentre, where some of the needs were discussed. + +g) I do not think that *graph* is the best thing to have. One would need a +sort of *matroid*, ie. a mathematical sound structure which keeps trak of +all sorts of *dependencies* (assuming that independent objects/data are +unrelated) Matroids allow you to keep trak of a minimal set of relations +between objects etc. (You may think of a combinatorial geometry, where the +points/lines/... may or may not be related. + It might be foolish just to have nods and edges, but one hast to +have an n-dimesnional structure of nodes, edges, faces, volumes, ... where +the semantic meaning could be attached to the *dimensionality* of the +object. This would allow to trace up and down iinto the complexity of teh +system if eg. code has teh smallest dimensionality (say 1 dim), algorithms +a hiher (say 3-dim (to let space for future enhancements)) and +documentation of algorithms or a proof it works etc even higher such +dimenionality. A *browser* would offer to display or hide the complexity +away from a user or show it to him. + Say a novice user will be confused to see all details but needs +quite practical help and may be even examples (like in the book) + +Sorry for not beeing as structures as you were, but I had no time to think +over Christmas ;-)) + +\start +Date: Tue, 30 Dec 2003 11:40:51 -0500 +From: root +To: bill.page1@sympatico.ca +Subject: [Axiom-developer] crystal and the semantic web +Cc: gilbert@sci.ccny.cuny.edu + +Bill, + +Sorry to hear about Faye's illness. I've been down for a week with +the flu and then useless for nearly a week with a bad head cold of +which I still have an annoying cough. It's a bad season for it. + +I worked on a combination rule-based program/knowledge-representation +program called KROPS (a combination of KREP and OPS5) years ago. It +had a lot of really nice features including automatic knowledge +classification. KROPS is an instance of a semantic network, thus my +interest in the subject. It has the unique property that rules and +knowledge chunks were dual representations. That is, you could write +in either style and update the underlying graph data structure. + +I'll look at the semantic web info you sent. + +\start +Date: Tue, 30 Dec 2003 17:58:05 +0100 +From: David MENTRE +To: daly@idsi.net +Subject: Re: [Axiom-developer] crystal and the semantic web +Cc: gilbert@sci.ccny.cuny.edu, bill.page1@sympatico.ca + +Hello Tim, + +root writes: + +> I worked on a combination rule-based program/knowledge-representation +> program called KROPS (a combination of KREP and OPS5) years ago. It +> had a lot of really nice features including automatic knowledge +> classification. KROPS is an instance of a semantic network, thus my +> interest in the subject. It has the unique property that rules and +> knowledge chunks were dual representations. That is, you could write +> in either style and update the underlying graph data structure. + +Have you a reference to a concise article descibing KROPS? Is this +software available as free software? + +> I'll look at the semantic web info you sent. + +I've started with : RDF Primer: http://www.w3.org/TR/rdf-primer/ + +\start +Date: Tue, 30 Dec 2003 18:33:14 +0100 +To: daly@idsi.net +From: David MENTRE +Subject: Re: [Axiom-developer] crystal and the semantic web + +David MENTRE writes: + +>> I'll look at the semantic web info you sent. +> +> I've started with : RDF Primer: http://www.w3.org/TR/rdf-primer/ + +And this sub-link of previous Bill's link is also useful: + http://www.w3.org/2001/sw/Activity + +\start +Date: 30 Dec 2003 17:27:46 -0500 +From: Camm Maguire +To: daly@idsi.net +Subject: [Axiom-developer] Speeding up compilation of EXPEXPAN.spad + +Greetings! I've looked at this a bit with si::*notify-gbc* set to t, +and am noticing an enormous amount of (apparently bignum) garbage +generation. (si::set-gmp-allocate-relocatable t) helps, but I'm +wondering where this is coming from, and whether a well placed +compiler optimization might eliminate it. Can someone point out the +time critical code snippet in lisp? + +\start +Date: Wed, 31 Dec 2003 00:35:16 -0500 +From: root +To: bill.page1@sympatico.ca +Subject: [Axiom-developer] crystal and the semantic web +Cc: gilbert@sci.ccny.cuny.edu, axiom-developer@nongnu.org + +Bill, + +I realize that I'll have to "sell" the crytal idea as it is new +and unknown. I've been trying to organize Axiom in various ways +to make it more useful in the long term and this idea "crystalized" +so I'm going to run with it for a while. Feel free to ignore me or +try to persuade me otherwise. + +More shower committee discussion leads to the following, somewhat +more "grounded" thoughts: + +COLLECT COMPILER INFORMATION + +Suppose we build a library of tools whose only purpose is to present +a slice of information. + +In particular, suppose we "instrument" the compiler so that it will +give us data about a given domain like: + +the domain name +the domain signature +the list of exports +the list of "required" domains +etc + +then we can use the compiler to automatically build databases about +the algebra. Axiom already does this with spad code (in the NRLIB +dirs). We can instrument the boot compiler in the same way. +We can even instrument the lisp compiler to dump information. +Once these databases exist and are automatically maintained we can +automatically build cross-reference and index tables. This information +can easily be gotten by using the "asq" command line program which +currently exists and queries the databases. Most of this machinery +already exists at the spad level but needs to be built at the boot +and lisp levels. + +A simple crystal facet would look at each of these lists (eg the list +of domain signatures). Related simple facets could (a) select a single +domain to inspect or (b) select domains by regular expression on the +signatures or (c) show the hierarchy of domains using this domain or +(d) show the hierarchy of domains used by this domain or (e) show the +exports, etc. + +Notice that all of this information is available from the compiler +databases and can be extracted by an asq command line program. +so we get + src -> compiler -> databases -> asq -> specific facet display +and since this process is driven from the facet we can merge +the crystal front end with a makefile middle and a database back end +so we get: + special information -> makefile -> specific facet display + +So where does that get us? Well, now we have a pretty little browser +that can walk all over the algebra. Useful but not very novel. The +key flaw is that it doesn't connect to anything except machine-readable +code. It does have the feature that we can write makefile-style +connectors between the database support code (asq) and the facets. +In fact, what defines a facet is two pieces of information: + the data that an asq command can extract + the relationship of commands required to construct facet data + +LATEX STRUCTURE INFORMATION + +Clearly we want more than just a fancy code-browser. Another need +is to connect the latex documentation to the code. Currently we +walk over a pamphlet document with noweb to create a stream which +can extract code from the document. And we have a higher level +function (booklet) that can assemble the pamphlet before extracting +information. + +We can easily create a "notangle" facet to show the code, a +"noweave" facet to show the latex (possibly in an editor like texmacs), +and a dvi viewer to see the tex output facet. We can even construct +a "booklet" facet that connects parts or whole pamphlets. + +But this leaves several huge and important gaps. Pamphlet files have +several semantically different sections. We need to recognize, parse, +and treat these sections differently. + +For example, in src/algebra/dhmatrix.spad.pamphlet we describe the +Axiom code with long sections of mathematics. We would like to +connect the mathematics with the axiom algebra in more than an +incidental way. Having them in the same pamphlet file makes at +least a marginal connection but we need to do more. + +If we look at the technologies available we could use several. +The easiest technique is to use latex tags such as \ref and \index +to mark terms that can be added to a database during search. More +generally we could invent tags that give us some connection between +the english text and the embedded algebra. + +The next technique is to try to "read" the actual text by machine. +There are also natural language parsing technologies (e.g. a chart +parser) that will give us some minimal information. I don't think +the technology is good enough to be useful. I have used this idea +in the past to read english text, form a "concept" of the sentence +and automatically classify the concept in a semantic network. This +would be easier for mathematical sentences rather than general +english as you could pre-populate the semantic network with appropriate +concepts from mathematics. + +The next technique is to require additional tags in the document. +This approach amounts to decorating the latex document with +type information. For example, instead of: +$x+y$ +we could add the axiom types: +$\type{x}{UnivariatePolynomial} + \type{+}{UnivariatePolynomial} + \type{y}{UnivariatePolynomial} +$ +which we could use to connect back to Axiom. This assumes that we +know the Axiom types which might not be true for most mathematical +expressions encountered in text. Perhaps this might be a useful +place to consider embedding OpenMath expressions. + +Another technique is to try to constrain the written expressions. We +can require that Axiom be able to parse expressions in pamphlets. That +way we can use the compiler to tell us what types are in the text and +how they connect to Axiom. This lends itself to automation with the +current tools. It also constrains the kinds of expressions you can use +in documentation to well-formed expressions. A fair number of math +expressions in books are not well-formed. + +The struggle here is to figure out a way to move from the latex math +to the axiom math and back again. Literate programs are a weak but +widely available technique. + +At a different semantic level, that of the whole document, we can use +bibliographic references to connect the documents. This could range +from hardly more than hyperlinks to more general semantic network links. + +SPECIAL INFORMATION STRUCTURE + +I'm advocating building an information source (database is really a +poor choice of words). It should be searchable using a semantic query +(parse and form a concept, "classifying" the concept and returning +nearest-neighbor concepts), using hierarchical structure (standard +hierarchical databases) and by keyword/grep/hashtable kind of +walks. Graph-relationship walks (like nearest-neighbor) should also be +possible. Standard relational queries are the least interesting way +to search this kind of information. I see this as a highly linked +graph object (a ball of string) with pointer tables to support other +views. + +GENERAL MATHEMATICS STRUCTURE + +Moving away from the specifics of a particular latex document we can +ask about facets which are "structural". We'd like to be able to see +the overall structure of parts of the world, usually in some sort of +a lattice. + +Lattice-style facets can be constructed (by machine) that detail the +inheritance hierarchy in Axiom and show the relationship of types and +categories. Lattice-style facets can also be constructed (by hand) +that show the relationship of the mathematics covered by the system. +Links could be constructed (again, by hand) that relate the real +concept (e.g. Ring) to the Axiom concept (e.g RING). If these were done +as semantic network "concepts" you could walk about the system using +the lattice. The various lattices could form the skelton of a concept +framework in a semantic network (our ball of string above). + +Indeed, the CATS (Computer Algebra Test Suite) eventually must extract +some sort of taxonomic organization of the mathematics used in the +test suites. This would be very similar to the NIST taxonomy for +numeric suites and would provide a concrete connection between the +mathematics and the available system functions. + +PERSONAL FACETS + +Eventually all of this machinery is justified by the need to organize +a very large (I'm assuming a 100x expansion of math in axiom) pile of +information so it can be used by one or a group of mathematicians to +do research. + +It needs to be able to accept dynamic changes (assume hundreds of +papers are published as literate programs every year and become +available at the rate of one per day) without human intervention. +I'm assuming that literate programs get published and become +available to global computational math research pipeline which can +be reviewed and searched. + +Crystal needs to be able to have several "research" threads available +at any given time. That is, I need to be able to highlight, remember, +collate, and organize existing work in multiple areas at the same +time. I want to be able to have several "papers" in progress and I +want Axiom to remember that I've visited certain areas of math +(e.g. crytography using group theory). I want the system to strip +out this part of math and Axiom and present it to me on one or +several facets so I can work on it. Eventually I want MY Axiom to +"know" (in some mechanical sense) what kinds of math I find interesting, +to search for literate papers related to that kind of mathematics, and +to extend the system to cover published results by selecting and embedding +literate programs into my copy of Axiom. + +The more I work with my copy of Axiom the more specific it should become +for me. The internal semantic network should eventually develop very +clear, specific meanings (concept clusters) for terms and ideas that +are important to me. Facets should be able to be dynamically created +based around these dynamically developed concept clusters. + +(As an aside eventually this clustering of concepts makes it difficult +for two Axiom systems to communicate. Consider an example of an electrical +engineer and a computer programmer. Both use the concept of a "register". +Eventually the EE's system has a concept cluster around the term "register" +that includes ideas of NAND gates, transistors, CMOS, ECL, etc. Eventually +the CS's system has a concept cluster around the term "register" that +includes ideas of register windows, base registers, stacks, etc. Notice +that the same "word" has completely different meanings to the two systems. +They will eventually become so specific that the concept nets cannot be +merged. Perhaps this is why it is so hard for geeks to talk to women :-) ) + +Well, I've wandered off into the weeds and it's time to work on something +useful. + +\start +Date: Wed, 31 Dec 2003 00:36:41 -0500 +From: root +To: camm@enhanced.com +Subject: [Axiom-developer] Re: Speeding up compilation of EXPEXPAN.spad + +You're seeing bignum computation during the compile of EXPEXPAN? +What a wildly curious observation. I can't imagine why bignums +are even involved in compiling. I'll look into it. + +\start +Date: Wed, 31 Dec 2003 00:53:39 -0500 +From: root +To: david.mentre@wanadoo.fr +Subject: Re: [Axiom-developer] crystal and the semantic web +Cc: gilbert@sci.ccny.cuny.edu, bill.page1@sympatico.ca + +David, + +KROPS was an effort I did while at IBM. I worked on two separate +research areas, expert systems which involved the OPS5 rule based +programming using a Rete. This eventually became a product (that +sold 2 copies) called Eclipse. The second effort was in Knowledge +Representation involving KREP, a research effort led by Eric Mays. +This eventually became a spinoff company. + +I found a unification of the two ideas, a Rete pattern-match network +and a semantic network. I created a language called KROPS (KREP-OPS5) +which had the interesting property that rules and concepts were dual +representations of the same object. So concepts could be classified and +expressed as rules and rules could be entered and classified as concepts. +IBM used KROPS as the basis for the FAME system (Financial and Marketing +Expert). FAME died and KROPS went with it. + +I have no idea what became of the software. + +The only published report on KROPS that I helped author was: + +"Integrating Rules and Inheritance Networks in a Knowledge-Based Financial +and Marketing Consultation system", T. Daly, J. Kastner, E. Mays, in +Proceedings Hawaii International Conference on System Sciences, HICSS-21, +Hawaii, Jan 1988 + +\start +Date: Wed, 31 Dec 2003 02:53:05 -0500 +From: root +To: Bertfried.Fauser@uni-konstanz.de +Subject: Re: [Axiom-developer] documentation and the crystal +Cc: gilbert@sci.ccny.cuny.edu, bill.page1@sympatico.ca + +>a) To me its not even clear how to structurize mathematics. One cxan +>build mathematics on sets, hence the Bourbaki approch, or even better (in +>my eyes, but equivalent in formal strength) on the *function first* +>principle. What to choose for axiom? (In fact the set approch is build-in) + +Well, I am assuming that we're only trying to structure "computational" +mathematics. That is, we're only trying to figure out how to organize +the pieces of mathematics we can compute. The NIST organization in the +U.S. government did a similar organization of information for numerical +mathematics years ago. Thus you can now find a specific index to classify +a numeric routine that will do Runga-Kutta integration on a certain class +of functions. A second index will find routines that do Simpson integration. + +Since we've limited the mathematics to computational forms we should be +able to collect algorithms and classify them just as NIST has already done. +Thus we would find different classifications for Clifford-algebra algorithms +vs Hopf-algebra algorithms. + +>b) I do not see how you can automatically assign semantics to data +>strutures etc. I think, one has at least to have a sort of *semantic +>typing* during the documentation of axiom (code). Hence every piece of +>documentation should come with a semantic type (or multiple such types) +>which finally allow to put a direction into the crystal looking glass. + +Indeed assigning semantics is hard. Three approaches leap to mind. +The first is by keyword assignment in pamphlet files. This is weak but +easy (although time consuming). The second is to use the compiler to +try to parse the mathematical expressions in the tex file and assign +meaning (types) to the symbols. This is hard but might be helped +along if we use OpenMath representations of the mathematics. I'm not +sure if OpenMath is strong enough to handle general mathematical +expressions. The third is to use a chart-parser and semantic network +software to try to read and classify the mathematics. + +I did an effort similar to the third case while at IBM. We built a +system called "Susan" which read english-language email, parsed it +using a chart-parser, constructed a semantic "concept", classified it, +and used the nearest-neighbor concepts to help direct further parsing. +Eventually the whole email and it's paragraphs, sentences, and phrases +became concepts. Once the email was classified (e.g. does it set up a +meeting? does it require an answer? does it have a deadline? is it from +someone important?) the email was assigned to a "basket". So email with +a "deadline" went into a "tickle file" that would begin reminding you +of the deadline several days in advance. + +We could build a Susan-like system that could handle some portion of +the mathematics because we have the advantage of working in a limited +domain. Since we know the domain we can prepopulate the semantic +network with concepts. These concepts can then be used to direct the +chart-parser toward a correct parse of a sentence. This would be great +fun but falls under the "real research" category and thus will never +get funded :-) + +>c) Algorithms should be plain and readable and not only be available in +>code form (if even this way) + +I absolutely agree. Indeed I would hope to see several versions of +explanation for algorithms. My current "best practices" example is +"Primes is in P" by Agarwal, et. al. +(http://www.cse.iitk.ac.in/news/primality.pdf) They have the closest +thing to a literate program that embodies the kind of information I +hope to see. They show the theory, the pseudocode, the bounding +conditions, the proof, a complexity analysis and references. I only +wish they had reduced the pseudocode to real code and published the +original tex. Other than that I feel they've "set the standard" for a +good computational mathematics paper. + +>d) To me it would be much more natural to look at the documentation like a +>big database and ask SQL like questions. EG: +> +>> select from AXIOM algorithm where domain has commutative; +> +>or such. And then get a sort of (semantic? web?) document which allows to +>go deeper into an algorithm, eg, thet should be lionks to al faces of the +>crystal which make sense to look at. + +I agree that a large database will eventually underlie the whole of the +system. I'm not sure that a relational database is an entirely useful +model. Yes, it is complete but it is a very awkward way to structure +mathematical questions. A semantic network concept would allow you to +"fill in" as much as you know about something (e.g. domain, commutative), +"classify it", and then find all of the concepts that it dominates. +Some of these concepts would be the domains you want. + +>e) The system will not be more smart than its designers / users. I do not +>see how an automated method will derive anything beyong mere syntactical +>sorting. To be frak I have no idea how to reach the above needs. + +We're not trying to make it smarter we're trying to make it useful. +At the moment Axiom is just a huge pile that is nearly inert. It +includes primitive query functions but little else. If you are going +to use a large system (scale Axiom by 100x) you need to be able to +move around it in much more fluid ways. As a researcher you sometimes +wander into a library and browse thru books that might be related to +some half-formed idea. Crystal is an attempt to bring "wandering" into +the computer age. + +Crystal should do at least 3 things. First, it should richly classify +and cross-connect the various sources of information (source code, an +algorithm, proofs of that algorithm, a complexity analysis, pointers +to related algorithms, the domain of application of the algorithm, +required inputs and their types, explanations of the theory behind the +algorithm, published results using the algorithm, domains which use +the algorithm, boundary test cases, examples of its use, etc. each +of which could be a kind of facet). + +Second, Crystal should watch how you wander and work to taylor it's +answers to your interests. If I am looking at information on +cryptopgraphy and group theory I don't want to know anything about the +weakly related concepts (e.g. the probabilities in threat +matricies). I would like the system to remember what I've looked at +(so I can find it again), to remember how I got there (so I can start +from some point and go off in a different path), and to "suggest" +(nearest neighbor) things that are related but I didn't know (or +forgot) to look for. And I'd like a facet automatically created which +is related to the interest and possibly other facets related to the +details. After all, if I'm trying to write a paper I'm usually piling +up a bunch of related things I need as background. Computers are good +at keeping track of things and we need to organize the tracks. + +Third, it should be capable of finding things in the stream of +computational mathematics literature that I would find "interesting". +I'm assuming that computational mathematics will eventually be online +and organized in some computer reachable way. Hopefully as literate +programs. So Crystal ought to be able to constantly be looking thru +all of the recently published papers for anything that fits my idea of +interesting. Trivially this involves a keyword search of papers that +get published but could be much more complex. In this way I can build +up a local library of references that make my Crystal searches more +interesting. And since they are literate programs I'm also building +up the mathematics for my areas of interest. + +We're capable of doing portions of the first idea with the current +tools but it requires people who write code to make an effort to +make their work more accessible to the machine. We don't even have +a standard outline of what should be in a literate program paper yet. +We need to build a few papers, build some technology to exploit the +papers, rebuild the papers and the technology, etc. until we get to +the point where we find the system useful. + +>f) As a probably managable project, it would be of utmost importance for +>me (and other mathematically interested, who are stupid programers) to +>have just the functionality described above for teh algebra lattice. I had +>such an email with David Mentre, where some of the needs were discussed. + +In the very short term (6 months?) we're likely to have the algebra +lattice available. David has discussed various ways of displaying such +a beast (it has 1100+ nodes at the top level). I'm looking at ways of +building a computational math classification based on test cases from +several different computer algebra systems (CATS). The combination of +the machine-built algebra lattice and the hand-built classification +system should give us a simple prototype to play with. We need a +simple browser tool. I looked at leo which is an open source literate +browser but haven't decided how to leverage it yet. We need a backend +(database/semantic network). If I can remember the insight from KROPS +perhaps I can reproduce it. + +>g) I do not think that *graph* is the best thing to have. One would need a +>sort of *matroid*, ie. a mathematical sound structure which keeps trak of +>all sorts of *dependencies* (assuming that independent objects/data are +>unrelated) Matroids allow you to keep trak of a minimal set of relations +>between objects etc. (You may think of a combinatorial geometry, where the +>points/lines/... may or may not be related. +> It might be foolish just to have nods and edges, but one hast to +>have an n-dimesnional structure of nodes, edges, faces, volumes, ... where +>the semantic meaning could be attached to the *dimensionality* of the +>object. This would allow to trace up and down iinto the complexity of teh +>system if eg. code has teh smallest dimensionality (say 1 dim), algorithms +>a hiher (say 3-dim (to let space for future enhancements)) and +>documentation of algorithms or a proof it works etc even higher such +>dimenionality. A *browser* would offer to display or hide the complexity +>away from a user or show it to him. + +I like the idea of dimensionality (faces, volumes). I don't believe I've +ever seen such an idea applied to a semantic network before. In a semantic +network "concepts" (data structures) tend to be very near each other +(counting links) if they are "similar". Semantic networks tend to have +"clusters" of concepts that express a particular "idea". These clusters +form because an idea has many similar ways of being expressed so there +are many links. Different "ideas" will have relatively fewer links +between them. The closest connection to dimensionality is probably that +a "volume" can be a "open ball" formed around a cluster of concepts. +If you measure the number of links that cross the boundary of the +volume and minimize that number you can count the number of "concepts" +inside the "volume". Thus you can have "big ideas" (a really rich +cluster of concepts) and "small ideas" (a small cluster). + +I'm unfamiliar with the matriod idea. What is it? An n-dimensional +spreadsheet? + +> Say a novice user will be confused to see all details but needs +>quite practical help and may be even examples (like in the book) + +I expect novice users to spend most of their time in the examples facets +so we should probably be sure they work :-) + +>Sorry for not beeing as structures as you were, but I had no time to think +>over Christmas ;-)) + +Every so often I try to look toward the 30 year horizon to decide if we're +building for the future. I was too sick to type over christmas so I had a +lot of time on my hands to think. Now that I'm back to typing I'm trying +to "ground" the ideas in working code. + +Shortly we'll all have access to several terabytes of disk, a terabyte of +storage, and a terahertz of cpu cycles connected to gigabyte bandwidth. +You'll be able to put virtually every paper ever published on a local +disk drive and search them in seconds. Imagine what will be possible +in 30 years. Libraries are doomed, books will be electronic. Browsing +will be incredibly painful if we don't get creative. + +\start +Date: Wed, 31 Dec 2003 03:00:38 -0500 +From: root +To: alexo@newmail.ru +Subject: [Axiom-developer] numeric cosine solution +Cc: axiom-developer@nongnu.org, daly@idsi.net + +>Hi! +> +>How can I solve some equation numericaly? For example, I need to find solution of cos(x)=x and I know initial guess value for x. +> +>Thank you. +> +>Alexei. + +x:=.33 +cos(x) = x ==> 0.9460423435 = 0.33 + +\start +Date: Wed, 31 Dec 2003 03:45:47 -0500 +From: root +To: david.mentre@wanadoo.fr +Subject: [Axiom-developer] Re: [Axiom-math] documentation and the crystal +Cc: gilbert@sci.ccny.cuny.edu, bill.page1@sympatico.ca + +>Usually, people have a top-down approach: they modelize a system +>abstractly (using more or less formal notations as UML or SDL) and then +>refine them through the actual code. But, in the case of Axiom, we need +>the reverse. We need to start from concrete objects (files, lines of +>source code) and add semantics to climb levels of abstraction. Of +>course, you follow different ladders, in the sense that understanding +>the compiler or the algebra would need different information and is +>structured differently, thus the different crystal facets of Tim. +> +> +>More concretely, I would propose the following approach: +> +> 1. start from parsers for the src/ directory. Parse directory structure +> and each file, categorize them (boot, lisp, spad, ...) and construct +> basic abstractions (list of lisp and boot functions; list of spad +> categories, domains and operators; ...) + +If you look at the int/algebra directory you'll see directories called +NRLIBs (e.g. DHMATRIX.NRLIB) which contain several files. These files +are output from the spad compiler. The databases are built from information +in these files. We can enhance the spad compiler to include additional +information in forms that can be written into databases or semantic +networks or whatever we like. The current databases (*.daase) files are +random access files. There is a C program called asq which knows how to +read these databases and answer queries. (src/etc/asq.c) + +> +> 2. from information extracted in step 1, construct one or several +> representation (knowledge graph for example) with the found +> semantics (name and body of a function for example), probably using +> a standard technology as W3C semantic web + +I'll have to look at Bill's references to see what technology they are +using for the semantic web. There are several dozen ways of doing this +and almost all of them are not "well-founded" in any mathematical sense. +KREP is well founded. That is, when you build a semantic network one thing +you want to do is put a concept into the network "in the appropriate place". +Most of these systems have heuristics for doing this. KREP has a logically +sound, well-defined predicate called subsumption for comparing two concepts +and deciding if one subsumes the other. In KREP a new concept is put in +the semantic network by starting at the top and using the subsumption +predicate to push the concept down until it hits "the right place". + +Semantic networks based on "is-a" links, or "kind-of" links and other +such random ideas are badly formed and suffer from a variety of logical +failings. + +> 3. the "Axiom explorer" (Tim, Bill or I) is interested in a specific +> "crystal facet" (e.g. the compiler). He builds another tool +> (e.g. call-graph analyzer for lisp code) which in turn is used to +> construct a new knowledge graph, concretized in additional +> information in the W3C semantic web. +> +> 4. Within the abstraction level build in step 4 (or 2), the "Axiom +> explorer" adds its own knowldege (e.g. this set of functions is used +> to parse Spad code, this other set is used for type analysis, ...) +> to the semantic web. +> +> 5. using previous abstraction levels and probably building a new tool, +> the "Axiom explorer" iterates, climbing abstraction levels, until he +> reaches his own goal. +> +>Repeat steps 1 to 5 with enough people to cover Axiom from A (Algebra) +>to Z (zerodim). +> +>Ok. I might be a bit optimistic and what I have said might appear more +>than abstract, but this is the current state of might thoughts. :) +> +>Even if you do not like above ideas, I think following "principles" are +>needed for a documentation system for Axiom (principles already +>formulated by Tim is his first email): +> +> o separation principle: in engineering, people separate a complex issue +> in _independant_ sub-issues to be able to understand them and solve +> them independently. We probably need to deconstruct Axiom in +> independent (or a least loosely connected) sub-systems to be able to +> understand them (i.e. the different crystal facets of Tim); + +I stole this principle from the unix world. use simple tools (asq, makefile) +and gang them together to make more complex tools. Thus I see asq-style +programs triggered by clicking on a facet to answer a question. Some facets +might be backed by things like texmacs, xdvi, or latex. + +> +> o "build on giant shoulders" principle: we need a way to reuse +> knowledge from other "crystal facets". For example, I would use Tim +> knowledge of the internals of Axiom to understand how the compiler +> compiles a given portion of the algebra; + +5'8" hardly qualifies as a giant :-) + +> +> o automation principle: Axiom is too big to add information manually on +> each function, each object, etc. We need tools to annotate a set of +> objects given a selection criteria (e.g. all operators in this Spad +> domain). + +I'd like to automate as much as possible but the automation curve +gets steep in a hurry. At least for the algebra code we already have +tools in place. If we can get to where we can draw a lattice in the +next 6 months I should have the data (both hand and machine generated) +for the algebra. + +> +>New year wish: I'll try to write and "show you the code" for above +>ideas. :) My own todo for this subproject of Axiom would be: +> +> - learn more about W3C Semantic Web (thank you Bill for the pointer) +> +> - find or write tools to manipulate Semantic Web (it might be Emacs +> with a proper mode or a more elaborated graphical tool) +> +> - apply above approach, starting from directory structure in src/ +> directory of Axiom. +> +> - from this first experiment, think about what would need to be +> "standardized", like common dictionnary or vocabulary, etc. Beyond +> usual technological issues, I think this point is one fo the harder +> point. How to build a set of knowledge that will be still useful in +> 30 years from now? + +>By the way, does anybody knows about a library to *draw* and manipulate +>arbitary graphs (in Common Lisp, C++, ML, ...) in a user interface? I +>know about DOT or VCG but they dot not match my needs: I would like to +>draw a graph, know when the user click on a graph node or edge and react +>accordingly. Does anybody know where I could find such an already made +>tool? Any knowledge of a browser for the W3C semantic web? + +There used to be a tool that would allow you to move around in a graph. +It was used to give a clever desktop. As you moved over portions of the +desktop the things nearest the mouse pointer got larger and were centered. +I can't remember what it was called but I thing M$ used a similar idea +in XP with the toolbar. + +I'm going to try to get ahold of a tool that runs in common lisp and +gives access to the gtk bindings. We can use it to prototype some basic +functionality (like opening an empty window, drawing a graph in the window, +embedding texmacs in the window, etc. The developer will be back on the +5th and I'll look into it further at that time. + +\start +Date: Wed, 31 Dec 2003 12:58:33 +0100 +From: David MENTRE +To: axiom-developer@nongnu.org +Subject: [Axiom-developer] More random thoughts on the crystal: goals & technologies +Cc: gilbert@sci.ccny.cuny.edu + +Hello, + +Thank you Tim, Bertfried and Bill for detailed replies. It makes me +realize that, despite I have said before, we have slighlty different +short term objectives. I try to summarize the discussion up to now. + +1) Goals + + A) Short and Mid term goals + + As far as I have understood, we have two short/mid term goals: + + a) (Bill, Tim, Bertfried) Document the algebra to have a kind of + reference manual for the Axiom programmer and researcher. Bertfried + has detailed the kind of Type browser he would like to have in the + short term + (http://mail.nongnu.org/archive/html/axiom-developer/2003-12/msg00042.html). + + b) (David) Document the compiler internals. I would like to start by + documenting the usage for the various files found in the src/ + directory. + + + B) Long term goals + + We all want a system to navigate into _structured_ information that + document the whole Axiom system. We all agree that you have different + views on such a big system and, depending on your current work, we want + to look more specifically at a subset of those views. Tim is using the + idea of a Crystal, with several facets. Tim has detailed some facets + examples: + http://mail.nongnu.org/archive/html/axiom-developer/2003-12/msg00056.html + + Now the hard part: which structure? How to document? How to explore? + What underlying technology? + + Regarding structure, I have proposed to use a graph like approach. Tim + has expressed interest for a Knowledge Network with a sound underlying + technology (KROPS). Bertfried has proposed to use a Matroid + approach. Tim and Bertfried objections is that we need a _structured_ + way of representing semantics in Axiom. I agree with them. + + + Regarding ways how to add information to current Axiom, several + approaches have been proposed by Tim + (http://mail.nongnu.org/archive/html/axiom-developer/2003-12/msg00069.html). + In summary: + - COLLECT COMPILER INFORMATION: use compiler knowledge to produce + indexes and other useful information + - LATEX STRUCTURE INFORMATION: add manual information (issue: + difficult to parse using a computer). Already used by Tim to + document the build process + - SPECIAL INFORMATION STRUCTURE: building "information sources" on + specific topics. Several sources for different purposes can be made + - GENERAL MATHEMATICS STRUCTURE: impose mathematical structures on + facets + - PERSONAL FACETS: the user builds its own vision of the system, using + information available in other users' system (it reminds me of + SmallTalk Environment) + + + From those several ideas, I deduce that we will probably need several + information sources (i.e. like databases but possibly with a different + model that the relational model). Those information sources should be + able to be parsed by a computer, but sometimes it won't be possible at + full extent (e.g. the PhD included in DHmatrix: we can extract some + literate programming structure but the full meaning of the english + text cannot be understood by a computer). + + We will also need several tools (Unix approach) to pick up information + is those several sources (and probably build other sources). The main + issue is to not use too many technologies, languages, etc. + + +2) Technologies + + a) W3C Semantic Web + + Bill as proposed to consider the W3C Semantic Web, which is build + around XML technologies like RDF (Resource Description Framework) and + OWL (Web Ontology Language). Those technologies are probably not well + enough structured (in the sense that Tim and Bertfried want to + structure semantics added to Axiom) but they form a standard that we + could build upon. Even using unstructured RDF, we could _impose_ our + own structure (e.g. Bertfried's Matroid) on our RDF files. + + Another important point in that they are build from the ground-up to be + distributed and reusable by any user. + + Those technologies are not very easy to use (e.g. cumbersome syntax) + but we could use them as a common file format to store our knowledge on + Axiom (Tim's information sources). It would allow use to build on + existing tool and infrastructure, to not reinvent the wheel and to + maybe to gather additional developers and community interest. + + For example, it exists a lot of research on ways to browse RDF files + (like MIT Haystack if I remember correctly). + + + b) Ad-hoc information sources + + Tim as suggested that we instrument the compiler to produce additionnal + information used to build information source. Using current tool (ASQ), + we could query those information. The advantage is that the technology + is already there. The disavantage is that the file format is + proprietary to Axiom, so we will need to build bridges with other tools + and technologies (browsers, query languages, ...). Bu I agree with Tim + that it can be used (1) as a first step and (2) as a proof of concept. + + + c) literate programming + + Noweb is useful as a first step to add information onto existing + code. However, we need to find a way to categorize this information and + build relations. We need a multi-dimensional book. Should we + standardize the naming of pamphlets (<>) or should + we add another technology (RDF?) above or below code and documentation + chunks? + + + d) "browsers" + + "Browsers" (in a wide sense, i.e. more than just showing hyper-links) + should allow users: + + - to add information is both a distributed (simultaneously in France, + Germany, Canada and US) and structured way + + - to "navigate" into this information, using both graphical approaches + (e.g. follow hyperlinks) and computational approaches (e.g. query + languages) + + They should be build on existing tools and technology (TeXmacs? + Haystack? Web browsers & HTML?). + + + e) Knowledge Network + + Tim has expressed interest for a tool like KROPS + (http://mail.nongnu.org/archive/html/axiom-developer/2003-12/msg00071.html). + It would allow users to put semantics in the computer in a sound + way. Possible issues: is it difficult to build such a system? How to + interconnect it with current and future information sources? + + + +3) What to do? + + I think our community has setup the goals the short term goals: + + - fix our current bugs (just a reminder ;), have a documentation and + release our first public release of Axiom + + - build a type browser as Bertfried request + + - build a usable lattice of the algebra + + - (for myself, I would like to also document the files in src/) + + + After those steps, we could make a point (a kind of workshop), draw + conclusion and ways towards the next step. + + + To decide on the technology to use, we need a consensus. Personnaly, I + am open to any technology, programming language (please avoid Java :), + etc. For the short term, we could probably consider using existing + information sources and tools (asq) and then consider a migration path + towards more standardized file format (RDF?). + + What is your opinion? + + + + + diff --git a/changelog b/changelog index 83d6095..47c8129 100644 --- a/changelog +++ b/changelog @@ -1,3 +1,5 @@ +20140420 tpd src/axiom-website/patches.html 20140420.03.tpd.patch +20140420 tpd book/2003-12.txt regularize 20140420 tpd src/axiom-website/patches.html 20140420.02.tpd.patch 20140420 tpd book/2003-11.txt regularize 20140420 tpd src/axiom-website/patches.html 20140420.01.tpd.patch diff --git a/src/axiom-website/patches.html b/src/axiom-website/patches.html index c7811e5..a65768f 100644 --- a/src/axiom-website/patches.html +++ b/src/axiom-website/patches.html @@ -4284,6 +4284,8 @@ book/2003-09.txt regularize, remove, reorder CC lines book/2003-10.txt regularize 20140420.02.tpd.patch book/2003-11.txt regularize +20140420.03.tpd.patch +book/2003-12.txt regularize