% iblechbot has joined #parrot % iblechbot has left iblechbot!~iblechbot@170.4-dial.augustakom.net % AndyA has joined #parrot % Senaka has joined #parrot seen chromatic chromatic was last seen on #parrot 1 day and 2 hours ago, saying: Or jonathan. [Apr 4 22:49:14 2008] seen Allison Allison was last seen on #parrot 4 days and 12 hours ago, saying: Infinoid: it's not currently spec'd. In fact, we're not planning to implement a compacting or copying collector until after 1.0 [Apr 1 12:41:04 2008] oops seen Infinoid Infinoid was last seen on purl 3 hours and 4 minutes ago, saying: seen Jonathan Jonathan was last seen on #parrot 14 hours and 33 minutes ago, saying: I thought the .c extension triggers this? seen lathos lathos was last seen on #parrot 2 days and 18 hours ago, saying: For that you may want the concurrency PDD. That's another one worth staying away from. [Apr 3 06:26:49 2008] seems that no one is arnd :( % Senaka has left Senaka!~senaka@124.43.220.157 % Senaka has joined #parrot % Senaka has left #parrot % iblechbot has joined #parrot r26801 | fperrad++ | trunk: : [readline] : - fix configure on MinGW32 diff: http://www.parrotvm.org/svn/parrot/revision?rev=26801 % Senaka has joined #parrot anyone knows about copy-on-write usage? % barney has joined #parrot r26802 | jonathan++ | trunk: : [core] Make Integer and Float PMCs handle somewhat better when subclassed by a high level object. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26802 % AndyA has left AndyA!~andy@87.238.46.251 % AndyA has joined #parrot % AndyA has left AndyA!~andy@87.238.46.251 % AndyA has joined #parrot % Senaka has left #parrot r26803 | bernhard++ | trunk: : [coding_std] : Satisfy c_operator.t diff: http://www.parrotvm.org/svn/parrot/revision?rev=26803 % Senaka has joined #parrot r26804 | bernhard++ | trunk: : #50596: [PATCH][PCT] Add basic "tests" : Apply patch from aparker42@gmail.com with some tweaks. : 'svn move' t/compilers/past-pm/*.t' in order to preserve history. : Load 'dumper.pbc' as 'Dumper.pbc' was not found. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26804 % rdice has joined #parrot % Senaka has left Senaka!~senaka@124.43.220.157 % Senaka has joined #parrot what is Parrot_dod_free_pmc for? have you got to explicitly say what to free? Senaka: Sorry, I never dug into GC % Senaka has left Senaka!~senaka@124.43.208.130 r26805 | bernhard++ | trunk: : Give credit to Andrew Parker. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26805 % kid51 has joined #parrot % barney has left barney!~bernhard@dslb-084-058-133-109.pools.arcor-ip.net % paco has left paco!~chatzilla@139.Red-80-36-122.staticIP.rima-tde.net r26806 | infinoid++ | trunk: : [src] Apply C++ correctness patch from RT #52510. Senaka++ diff: http://www.parrotvm.org/svn/parrot/revision?rev=26806 Infinoid: Have begun looking at the ops2c patch. This might be a good candidate for starting a new SVN branch in which to do revisions -- particularly if changes to the tests are needed. ok. what's the goal? the goal is to announce a boostrapped perl6 one way or another in YAPC::NA :) purl: I think we've already achieved that goal. kid51: sorry... I'm happy to work on refacting ops2c further, but it helps me to know when to stop :) Since ops2c.pl is invoked very early in the 'make' process, any errors we make have big effects downstream. So this limits potential damage to others working in trunk. This wouldn't have to be a long-lived branch, but it would permit all (3) interested to work on it without bothering others. sounds reasonable It's what I did when I refactored this script originally -- but I'll let you make the call. please go ahead and make the branch k I guess my first step would be to document Ops2c::Utils, because I'm unlikely to understand the code otherwise. Call: svn co https://svn.perl.org/parrot/branches/ops2c r26807 | jkeenan++ | ops2c: : Creating ops2c in https://svn.perl.org/parrot/branches diff: http://www.parrotvm.org/svn/parrot/revision?rev=26807 r26808 | jkeenan++ | ops2c-26806: : Tagging trunk at r26806 so that the ops2c can later be synched to it. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26808 Ah, this dalek bot fellow performs much better than svnbotl did in this usage. dalek++ seen barney barney was last seen on #parrot 1 hour and 9 minutes ago, saying: Senaka: Sorry, I never dug into GC exterminate all humans I think svnbotl ignored all changes to tags/, only reporting changes to trunk and branches this one isn't so discerning. I don't mind that. Tags don't change very often. But in this circumstance, svnbotl would puke at the large number of files being created/modified and go off-line for 10-15 minutes. Fixing it to ignore tags would probably be a snap, but might not be worth the effort. * kid51 applies Infinoid's patch to Ops2c.pm in ops2c branch and runs tests with Devel::Cover All tests in t/tools/ops2cutils/*.t continue to pass. Coverage actually improves slightly: lib/Parrot/Ops2c/Utils.pm 99.2 86.1 100.0 100.0 maybe I reduced the number of conditionals slightly. *shrug* Re-reading the POD in Parrot::Ops2c::Utils, I see that I had to write an FAQ to explain some of its arcane aspects. The most unusual/annoying aspect of it was the closing of a filehandle and its subsequent reopening. I've never seen that in any other Perl program. I can see why you'd be so enthusiastic about refactoring it. r26809 | jkeenan++ | ops2c: : Applying Mark Gline's 20080405 patch in : http://rt.perl.org/rt3/Ticket/Display.html?id=52506. Refactoring of several : internal subs in Parrot::Ops2c::Utils. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26809 * Coke wakes up. r26810 | jonathan++ | trunk: : [core] Make scalar PMC (and things inheriting from it) work better when inherited from by high level classes. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26810 r26811 | jonathan++ | trunk: : [core] Some of the key handling code blindly assumed what it was being passed was a key. Add some checks for this. Before, the fact that you passed an Integer PMC into there and it would do something sensible (as happens with array access) was pretty much blind luck. This change lets languages that have different integer types that maybe use flags or pmc_ext have their arrays work. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26811 % paco has joined #parrot % jan has left jan!~chatzilla@90-229-128-103-no52.tbcn.telia.com % Senaka has joined #parrot seen Infinoid Infinoid was last seen on #parrot 51 minutes and 48 seconds ago, saying: I can see why you'd be so enthusiastic about refactoring it. seen Jonathan Jonathan was last seen on #parrot 21 hours and 16 minutes ago, saying: I thought the .c extension triggers this? jonathan: ping infinoid: ping I need to know few things regarding the GC. Is anyone around? r26812 | pmichaud++ | trunk: : [rakudo]: : * Fix 3e4 parsing bug (RT#49910). diff: http://www.parrotvm.org/svn/parrot/revision?rev=26812 Senaka: hi hi What do you need to know? I'll answer if I can... k thanks first of all have you got to call methods such as, Parrot_dod_free_pmc? Isn't the GC supposed to internally manage these? No, those should just be called by the GC itself. next is that, as it seems from the exposed interface, the GC is un-aware of object allocations? If not, the interface seems to be distributed beyond the header dod.h Allocations are out of arenas. I think it's maybe memory.c that deals with some of that. so isn't the GC informed saying... I'm picking space from you? maybe the GC works that out after the fact, during the next sweep Right. It goes through the arenas that are allocated. you mean the allocation? No, the GC. yes The allocation is done out of these arenas. :) the GC discovers the allocation after it's already happened I meant the discovery rather ok but is this inline with normal GC operations? ex:- say I allocate large chunks so rapidly and free them almost instantly so, the "sweep" phase discovers a lot of newly allocated blocks or doesn't :) sweep isn't called that often isn't it? and the GC being stop-the-world I don't think so or the mark phase am I missing something here? sweep is called either explicitly from the bytecode using the sweep instruction, or on an allocation when all arenas are full. * Infinoid still learning the terminology "or on an allocation when all arenas are full." where is the code that figures this out? grepping didn't get me any clue... :( src/gc/memory.c I think oh no, not there src/gc/smallobject.c That one. cool found it what's get free object doing? do you reuse dead PMCs? hmm seems... seems? I know not seems. it'll reuse the buffer from them, sure While everyone's talking about GC, how is the GC going to handle an embedded C library that does it's own allocs -- will it take hints from the shim code that does the embedding, or just let that all play off in a corner on its own? malloc can be wrapped right? The GC only cares about objects allocated out of its memory pools. stuff allocated outside the arenas can fend for itself Right. yes that's legal new question where would such objects lie? you're asking where PObjs are stored? Infinoid: stuff allocated outside the arenas can fend for itself doesn't the Arenas span the heap space? oh. that'd end up wherever malloc decides to put it, which is platform-specific I see The areans hang off struct Arenas *arena_base; in Parrot_Interp structure (interpreter.h) * skids assumes if a PDC references into such an allocation there is a way to tell the pointer apart from a GC-controlled buffer object? skids: it wouldn't be pointing into an arena's memory range so ok... Now since allocations are handled by Arenas the GC will have to rather seek locations of such allocations perhaps scan a list on Harmony, the allocation would happen through the GC rather so will have to work on some wrapper perhaps or are their some callbacks etc.? Senaka: Harmony can service the calls parrot makes into smallobject.c, right? you wrap those, I mean are you talking about object allocation, or arena allocation? object ok. so yeah. parrot calls gc_ms_get_free_object (or whatever), your wrapper calls the appropriate Harmony function, and Harmony gives us a newly allocated buffer right? So when reading the new PDD I did think of a potential "confounding case" surrounding the "buffers never point to PDDs" thing -- a parrot-native trie data structure would have trouble with that, given that "keys" and "values" (the latter pointers to PDDs) are stored in the trie. (and having a mapping table of values to PDDs would take extra space, defeating the purpose of a trie) PDDs? PMCs? kid51: you around? Yes newly allocated buffer ... chunk of memory rather Infiniod: right, sorry, still learning to squawk right. :-) well, may be we'll have to address PMC allocations and buffer allocations separately is it? I was under the impression that PMCs resided within buffers hmm buffer is inside PMC ain't it? kid51: how goes the TAP::Harness refactoring of t/harness? I'm volunteering the rest of my day today to set up a build farm, using whichever (smolder/pgbuildfarm) build farm orchestration we decide(d) on. that's what I was thinking I have 400GB of disk and 3072MB of ram and VMWare Server 2.0 beta2 on amd64 to devote to the project. from PDD: "Parrot really has two independent GC models, one used for objects (PMCs) and the other used for buffers (including strings). The core difference is that buffers cannot contain other buffers, so incremental marking is unnecessary." diakopter: that's one good box It' skids: yes it says so that makes sense. there's some confusion over the definition of "buffer" PMCs are PObjs It's been rather dormant. But I have to eat breakfast now, so let me get back to you in a while. buffer is like any free space? your "trie" data structure is a PMC, which points to other PMCs % kid51 is now known as kid51_afk % iblechbot has left iblechbot!~iblechbot@ppp-62-216-204-111.dynamic.mnet-online.de data buffers (strings and so forth) are allocated from a different pool, I guess yes two arenas pdd09 says so Infinoid: internally it has to point to other buffers -- every node in the trie being a PMC would not allow for a compact trie, too much overhead. * diakopter begins creating oodles of VMs. ok, now there's another concept of movability of buffers so I believe that a buffer == raw chunk of memory isn't it? (mainly I came by this quandry when pondering fixed-type-hashes-as-tries and libjudy, but if it turns out that regexps can do double duty as efficient DFAs that might be better than a trie implementation) Senaka: in this case, "buffer" has the specific meaning of "chunk of memory allocated from the 'buffer' arena, used for raw data and not for PMCs" Infinoid: yes e.g. adding a pair to a "trie" is $regexp = regexp { $regexp | $pair }, and dumping the trie uses the :e adverb. so I think I got it right here ok :) so well that seems sorted and perhaps might be too early to decide on what could be done what could be done further what needs some attention is how the GC discovers the allocation? isn't that what the whole mark & sweep thing does? hmm... well ok let me put it like this in Harmony, when you allocate, the GC can be made aware of the process, so that it will Garbage collect if the memory is exhausted how does Parrot do that? smallobject.c doesn't say much on this fact are you talking about the need to allocate more arenas, or the need to exert memory pressure because the system is out of RAM and malloc failed? like, isn't it required that the GC running in some situations where there is no contiguous blocks of memory at least in the arenas for buffers sure, so it allocates a new arena and adds it to the list new arena? I think you didn't get me say that I want to allocate in a given arena. But it is nearly full. freespace is scattered. Now when I call alloc, should the GC get called too so that it can make space for me? alloc is a gc function, so the gc is called if the arena is out of free objects, it needs to either call malloc() to get another arena, or realloc() to expand an existing one, right? hmm ok well in the case of PMCs yes in the case of buffers not always you can compact ain't it? if you find all the pointers pointing to those buffers, and fix them all up, and prevent contention from other threads, you can move things around, yes I don't think we do that at the moment. aha that's what I was searching I guess :) on which OSes/architectures do folks want to see parrot smoked nightly this is where we start talking about bounce pointers or backreferences or read barriers, and people's eyes start glazing over ok seems sorted next issue is regarding Copy-on-write Harmony doesn't support this is COW mandatory? COW is a memory optimization. you could just copy immediately, at the expense of extra memory usage and extra processing overhead I don't know how big an effect it would have. (I've seen instances where the effect would be huge, but I don't have any real world stats.) ok so perhaps it is not a key requirement isn't it? it is not mandatory its just a nice hack, where it's available. k cool Now I have just a few left to sort out Parrot_dod_trace_ methods are some we'll probably have to fix up some code which expects to set the PObj_COW_FLAG bit and borrow another object's pointers, but that's all hmm % jan has joined #parrot % tetragon has joined #parrot Parrot_dod_trace_roots will mark all live objects isn't it? looks like it trace_system_areas() is interesting, traces the current thread's stack that's thread stack enumeration :) ./breakfast & ok % buildbot has left buildbot!~buildbot@smtp.matisse.net any thoughts on pinning objects? % kid51_afk is now known as kid51 diakopter: Broadly speaking, there are two big problems with our smoke testing: * Limited scope of platform/OS/C-compiler combinations from which smokes are being submitted. * Badly designed software which refuses to accept valid smoke reports but does allow people to post raw HTML directly to our servers. Obtaining a big server with plenty of VMs helps with the first problem, but not necessarily the second. I started to refactor the various components: * t/harness * lib/Parrot/Harness/Smoke.pm * tools/util/smokeserv-client.pl * tools/util/smokeserv-server.pl But ran into several obstacles: * Parrot::Harness::Smoke require-s two obsolescent CPAN modules, Test::TAP::HTMLMatrix and Test::TAP::Model::Visual -- and then overwrites a key sub from Test::TAP::Model. Very much a hack. * Trying out changes in our smoke setup requires coordination with perl.org webmasters, which, understandably, cannot be a high priority on their schedules. * Coke wanted us to consider Smolder, Tinderbox, etc. The latter did not seem appropriate for our needs; the former is poorly documented. My hope was to refactor the 4 files above so that I could run a test smoke server on my own server and then: * Have t/harness and Smoke.pm generate a data file -- rather than a fully CSS-ed HTML file -- which would be sent by client.pl to a server of anyone's choice, which would then transform the data into HTML (or any other reporting medium). * Use this set up for fine-tuning and debugging. And that's where things have stood since Jan 23. .eor Senaka: apparently not Ok and is it possible to force GC? yes I think since the interface is exposed yeah, there's a pasm op for that t/op/gc.t tests it what the op? it has been said that the op is an opcode structure. It's the thing that perl -Dx prints collect http://svn.perl.org/parrot/trunk/docs/ops/core.pod purl, forget the op Infinoid: I forgot op oops, nm -- that one isn't in the repo :) yeah 404 http://svn.perl.org/parrot/trunk/src/ops/core.ops look for found it on the machine http://svn.perl.org/parrot/trunk/docs/ops/core.pod srry /usr/local/share/doc/parrot/docs/ops/core.pod right... docs/ops/core.pod gets generated from src/ops/core.ops afk, lunch got it Parrot_go_collect in parrot/resources also, can you time the GC operations? % Limbic_Region has joined #parrot kid51: I have a VPS which can host another smoke server, if necessary, helping the 2nd problem. Which smoke system do you want to use? Smolder? % iblechbot has joined #parrot I know very little about Smolder. When I went to its sourceforge sites, I couldn't find much documentation. So I wasn't in a position to explore it. If you are familiar with it and can get a prototype up and running, then go for it. Would Smolder both run the smoke tests and display the results? Or would it simply receive reports generated by average Parrot users and display them? Or some combination of the two? * diakopter looks at http://buildbot.eigenstate.net:8040/ I guess there is http://www.perlfoundation.org/parrot/index.cgi?buildbot does anyone know anything about that? This page is mostly my musings: http://www.perlfoundation.org/parrot/index.cgi?rfp_parrot_needs_better_smoke_reports kid51's url is at http://xrl.us/biy53 http://www.perlfoundation.org/parrot/index.cgi?buildbot is Matisse Enzer's description of that which you were looking at at eigenstate.net Both pages were last updated in January of this year. kid51: do you still think storing test results by testfile.t is the right way to go? (I'm not challenging it; just curious) r26813 | jonathan++ | trunk: : [rakudo] Start refactoring the type hierarchy somewhat. Add the Any type and make types such as Int, Num and so forth lie beneath this - basically, all but Junction. This should allow us to get MMD and Junctions working with Any as the default parameter type. Additionally, it means we can start making classes do various roles, like Positional. Due to what appears to be a Parrot bug, the vartype storage doesn't seem to get inherited in classes that also inhe diff: http://www.parrotvm.org/svn/parrot/revision?rev=26813 kid51: I think I'll set these up merely to run 'make smoke' and not try to solve other problems. What OSes/architectures do you suggest I create as VMs? I was going to do: 32- and 64-bit Windows 2003/2008 on cygwin, strawberry, and msvc++ (12 Windows VMs), and 32- and 64-bit DebianSid/Centos5.1. any other suggestions? diakopter: In debugging test failures, t would be useful for me to say, "At what point did this test start to fail?" That would seem to require a data structure where the fundamental unit is the individual .t file. Which is what it in effect is right now ... except that the data is not searchable because it's all hard-coded in HTML files. Those are excellent starting points. We don't get many reports from Windows in general, and we would benefit from regular periodic smoking on the various C compilers people use on Windows. % Senaka has left #parrot Some people get consistently smooth builds on Windows; others experience horrible failures. But we don't have enough data to spot patterns. It would be good to add a "diakopter's battle plan" section to http://xrl.us/biy53 wow, Ops2c::Utils->_reset_line_number() is silly. a simple seek(0, 0) would work much more nicely. Feel free to refactor away! As I said earlier, that's the way the code was before I started with it, and not breaking anything was my biggest concern. But in a branch, you can break as much as you want ;-) \o/ * Infinoid goes for the high score r26814 | infinoid++ | ops2c: : [ops2c] Stage 1 of cleaning up the silly file handling: : * single function to write a C source file, just calls the top/bottom functions : for now. : Next step is to fix the handling of $SOURCE, and to use seek() instead of : reopening it. Or just use $SOURCE->input_line_number() instead of counting : them manually. *gasp* diff: http://www.parrotvm.org/svn/parrot/revision?rev=26814 r26815 | infinoid++ | ops2c: : [ops2c]: : * name it print_c_source_file() to be consistent with print_c_header_file name. : * make print_c_source_file responsible for opening the file, then pass that to : the child functions. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26815 diakopter: all of them? Coke: sure, why not? diakopter: seriously, though, we seem to have linux and darwin covered. something solaris x86 would be awesome. Coke: ok Coke: oh, i see your joke I've used SunOS... does that count Infinoid: 26814 passed all tests. But tests start to break at 26815. diakopter: kid51 has some excellent points, but if it's a question of which platforms, pretty much anything other than linux will help more than linux. (just because many of developers use that and catch stuff sooner than later.) this is prep to fix _reset_line_number(), which I should have done in a few minutes diakopter: yup. % askie has left askie!~askie@81.171.100.207 r26816 | infinoid++ | ops2c: : [ops2c] It helps when I use the correct syntax for IO::File. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26816 amd64-linux-x86_64 is updated infrequently "kid51" at 70.107.12.156 pasted "Failures in 'make buildtools_tests' at r26815" (31 lines) at http://nopaste.snit.ch/12631 Coke: agreed re platforms there needs to be more google hits for "Parrot polishing" I just told a friend that's what I was doing today, and think I gave him the wrong idea. % kid51 is now known as kid51_afk % Coke is now known as Coke_oooh_ps3 % askie has joined #parrot ok. I'll have to fix up the test suite from the API changes I just made r26817 | infinoid++ | ops2c: : [ops2c] There, _reset_line_number() no longer reopens anything. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26817 * diakopter downloads solaris 10 isos r26818 | infinoid++ | ops2c: : [ops2c] Fix and clarify the SYNOPSIS a bit. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26818 % braceta has joined #parrot % ruoso has left ruoso!~ruoso@81.84.156.87 r26819 | infinoid++ | ops2c: : [ops2c] Fix POD to pass t/doc/pod.t. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26819 % ruoso has joined #parrot r26820 | infinoid++ | ops2c: : [ops2c]: : * Move the final rename() into print_c_source_file(). Child subroutines should : only be concerned with printing. : * Add a test for print_c_source_file(), based on 03-print_c_header_file.t. diff: http://www.parrotvm.org/svn/parrot/revision?rev=26820 % grim_fandango has joined #parrot r26821 | infinoid++ | ops2c: : [ops2c] update MANIFEST and metadata diff: http://www.parrotvm.org/svn/parrot/revision?rev=26821 % kid51_afk is now known as kid51 4 tests still failing as of 26821 kid51: any other platforms besides solaris? Well, why not ask for the moon and the stars! OpenBSD, NetBSD AIX kid51: fixing up t/tools/ops2cutils/ right now last time I tried dragonfly, it was at a very early stage of support Let me ask an OT question: If I've just been given a shell account on a Solaris box, and if the default shell on that is ksh, but I want to use bash, what is the quickest way to make bash my default shell? check if the chsh command exists. if it does, use that. Infinoid: We were getting regular smoke reports on Dragonfly; good ones too. great. I had to patch some stuff in config/ to even get it to build but that was a while ago. * diakopter wonders if AIX is available % Senaka has joined #parrot Infinoid: Unfortunately, no 'chsh' or similar commands on this Solaris box. Would setting up a .profile file work? to make ksh execute bash? probably might be as simple as inserting an "exec bash". but if it doesn't work for some reason, it might be difficult to remove that again. % braceta has left #parrot have a second shell open just in case u mean bash on solaris? Yes. or any other shell?