After a week long vacation , I finally started work today .In the holidays I wrote this initial draft of the protected mode tutorial.I shall speak to sir soon about his feedback regarding this.
Have given proffesor rkk , the demo twice , I think the coding part is mostly over , many changes , particularly in the command syntax were suggested and have been implemented.Also now system calls have been implemented , the following syscalls are there :
Released a tutorial which can be found here , along with the latest version (v1.0.0) , which can be found on the sources page.
Met sir on the 17th and discussed some of the features , command syntax , and had feedback on my documentation .I have now updated most of my web pages , and have also provided some screenshots.
Implemented some of the extra features with professor adivsed me to , details include the ascii display of the memory dump .Ability to scroll .The ability for someone to specify the address till which he wants to run a program,Ability to trap a fault in a user program program.
Fixed some more mem management and keyb handling interrupts. Wrote a list of features that ProDebug shall support . The list can be found here
Implemented the hex in code keying feature in which a person can just key in hex machine code , and then run it in protected mode. Also improved upon my online documentation further .
I was going through the documentation of the dos debug program and noticed a useful feature called gotill offset , in which you specify till which offset you want to run the code till .This is implemented using the software breakpoint (or the int3 (oxCC)) instruction , what happens is that the byte till which you want to go till is replaced with the byte 0xCC (int 3) , which is a software break fault for the 386.
Implemented the same feature on my debugger. Now only commands to copy,move,fill memory are left to implement . Ofcourse the dissassembly function , and the function where in a person can key in asm code (not hex) will not be implemented by me as that is a whole of project in itself.
Modified my code to put all the thread specific segments into the ldt. Added routines to display the ldt , and also added code to create that ReadOnly Segment mentioned before. Also did some work to enable the user to key in hex code. Also simultanously working on the help function (ie online help) of the debugger.
Preparing a writeup on the UI provided by the debugger ,should be done by tomorrow .
Met Proffesser today , to discuss his feedback about my prelimary demo. This is what he had to say :
Fixed those bugs , basically the problem was that since my cs was non conforming , my dpl as given in the cs's descriptor should have been equal to the cs's rpl . However this was not so , hence it was giving a problem . Also in another asm routine, I was popping of less no. of items from the stack than I was pushing at the beginning , this surely used to kill the debugger whenever that function was invoked.
Fixed some bugs , rolled out version 0.03, as required by sir , the images of the debugger and the test program have been kept in the source code area . Please note that currently a gpf is reported by the debugger , at instruction 0x1e , I am investigating . Added support for cls , Spruced up my makefiles and made some scripts to ease the creation of test programs .A list of currently observable bugs can be found in the file Bugs in my tar ball .
On this D Day , I am very near finishing my project . I fixed the earliear problem by simply doing away with variable jumps , Now i jump into a fixed location 01000h , which is hard coded .My main function begins from there . Also I made all my C code to pure ansi .Added functions to create a read only segment ,kill a program , change registers etc .And code to display the gdt .
Spent the better part of the day trying to port my code to dos . I am using djgpp and nasm . For reasons described below(earlier) . At the end of the day , I have some problems while building , which I will detail below .
I can compile and link everything . The problem is this , After setting up the protected mode from my boot up code , I jump to the initialization routine , called Main . For doing this I need its address , as this has to be hard coded as a far jmp . For this I had made a perl script for going through the linker produced storage map , to find its offset. it was something like this :
nasm -f binary -DENTRY_POINT=`perl ../scripts/findaddr Main` -o startup.bin . However in dos this is not possible , ie taking the output of one program as a command line argument. So I tried three approaches . The first one is to find the address myself , change the makefile to something like nasm -f binary -DENTRY_POINT=0x1734 . Which will obviously have to be done each time the code changes. This is obviously not acceptable.
My second approach was to try to output the perl's output into a batch file , of the form SET ENTRY_POINT=0x1734 , and then to invoke nasm like this nasm -f binary -DENTRY_POINT=%ENTRY_POINT% . However I tested my code in winXP and got a strange "Out of environment space error" , I tried various approaches and searched a lot for this , But presently am clueless about solving it(apparentrly solutions for win98 exist but they don't work on XP. The third method I am exploring is to install bash for dos , which also is not too attractive an option. I would be obliged if you could share your insights into this problem.
Did some digging around for converting my code , so as it compiles on windows machines also . However sadly I find that 16 bit tools like Turbo C/C++ ,Microsoft C,MASM,TASM are out of question . What we need as we are writing 32 bit code are 32 bit tools , the modern tools like MASM32, NASM,DJGPP,and Visual C++ should work fine. But now we are back to square one with windows machines. As most of these tools are likely to be not present , and hence have to installed. I am working on my code to work with dos versions of nasm and the dos gnu port of gcc djgpp .
I came to this decision after seeing that anyway any of the tools we need for example masm32 or tasm32 or even newer 32 bit versions of microsoft C , would anyway have to be downloaded and installed , and some of such tools aren't even free. I thought why not stick to the tried and tested gnu tools for dos. I need your suggestions for this .Please advice me on this.
Implemented the break support , now a person can define breakpoints , example breakonpc 2 , would break when the instruction starting at byte no 2 is executed ,breakonread 13 would break when the memory location 'ds:13' is being read , and similarly breakonwrite would break when the above location is being written to . Also added support to clear breakpoints.
Now working on breaking a user code when a general protection fault is raised , ie when he does something like modifying his segment registers , or trying to reference memory that he is not alloted , this should behave like the break command , ie the program should be breaked with a brief description of why he was breaked and a cpu dump should be provided.
Later that day ...
Uploaded my 2nd version of the debugger , which also supports debugging using breakpoints , Please go to the sources section to download it.
Removed all inline assembly from my code , I think this leaves my code in a position to compile on a dos machine. However the only 32bit freely available C compilers and assemblers I could find were Tasm(Masm is not free , although both are compatible),DJGPP (Gcc port to dos, hence pretty robust), Borland C++ (Again not free)and Microsoft C (not free, like all other Ms tools). So I plan to mostly use djgpp and tasm(or maybe also masm , since I have it).
Also note that due to the coming of end sems , I will be able to put only very few progress reports till they get over.Please bear with me.
Debugged the earlier error , where I used to get a general protection fault , This is a catch all fault thrown by the 386 for example when you try to execute code (ie the pc points to ) somewhere which is not a valid code segment . Or for example when you overstep the segment boundaries , or for example when the processor gets an interrupt whose interrupt gate does not exist. In my case It was a very silly mistake which took a lot of time to find out . I had defined some two memory constants wrongly(basically in decimal instead of hex) which took almost a day to figure out :( . Also wrote code for the break function using the debug registers .
Also I had met the proffessor yesterday and am working on the documentation of the boot up code . Question : Instead of writing the whole thing myself , can I include some excellent web references ?
Finally uploaded the first demo version of my code , right now It would compile only on a linux machine , having the standard devlpt tools like nasm,gcc,gnu make etc. Please check it out.
Came back from home today . I debugged my memory management routines , Also there was a routine in the trace through function , have debugged that also . Now I don't use much of dynamic memory allocation . The problem was that the memory allocators internal data structures were being written with garbage . So I shifted them to some other region and it worked .
Right now I get a general protection fault when I try to reload a program , I working on fixing that . Also working on writing handlers for all the exceptions that the processor might throw.
I had a hard time debugging my memory management routines , also my floppy driver does not seem to work with all the controllers , that is because sometimes some operations fire an interrupt on some controllers , whereas on others they don't . I will now reprogram it using timers , so that in worst case it can work with any controller , For this I will have to program the PIT , and write a timer interrupt handler . Also I observed some bugs in my scanf routines .
Going home today , will be back on 15th , was in a lot of hurry so couldn't meet the proffessor , will do that the first thing after I get back .
Wrote the main command loop which is responsible for taking the user commands and acting on them , Wrote the load routine , the loadAsInterrupt routine and the createSegment routine . Right now I am reading on how exactly to use the debug registers to issue a break point. One question I am pondering over is how to dissasemble the machine code , for this I am looking up the docs on the libdisassm , A disassembling library , I find lots of dependencies to libc functions there , so I think I will have to implement those too.
Prepared documentation about my project , which can be found here . Some of it is still incomplete and shall be updated regularly .Also thought in detail and prepared a list of features to implement , it seems the real fun shall start now :).
Finally , my floppy driver is running on a real machine , I wrote it fully from scratch . It can handle all sorts of errors and uses dma .However it uses busy waiting on interrupts instead of using timers.Right now it can only read one sector at a time.
Also I had speaken to proffessor rkk today , I will be updatng my documentation shortly.(by this weekend).
I have fixed the bug in the floppy driver which now works fine with bochs , ie returns the sectors properly , the bug was pretty simple , I was seeking to sector 0 , which does not exist , it starts from sector 1 , apparently :) . But now when I tested it today on a real machine , it did not give me the correct output of a sector , since I am not taking care of any errors or retrying while reading ,a false interrupt must have been given which caused me to return junk data , I am trying to understand the linux 0.11 implementation of fdc controller to see how to implement error checking.
Spent the larger part of the day trying to figure out why bochs chocked out my floppy seek command , It seems to choke on a particular byte given to the fdc , however according to the intel fdc manual , it looks right ,I really cluesless about this bug , because even the linux kernel 0.11 does the same steps as me . I haven't tried my code on a real machine , only on bochs , so this might be a bug in bochs ( a very very slim chance).
Fixed the printk bug , It wasn't scrolling properly, had to rewrite some low level video functions for this.Now working on putting scanf capabalities .
...Later that day , finished the scanf routine . Now on the floppy read debugging.Right now the routine just reads one block and puts it in a specified mem location . I have to make it read some continuous no. of bytes.
I am still debugging my first version , which tries to trace throught the program using the trace flag . Most of the bugs have been fixed . I discovered some really silly ones . Now I am working on this bug : The user program's first instruction gets executed , then my interrupt handler is called and I switch back to my main loop , then nothing happens and the screen becomes awfully discolored. When I tried running it in bochs the same thing happens . Debugging is really hard here , because all I have to help me are printf statements and I can't step thru the code. Guess by tomorrow It should be fixed .
... Later that day , problem fixed , the bug turned out to be 16bit pop , resulting in stack misalignment. So now I have tried out tracing thru a user program , although some doubts are creeping into me regarding what exactly my specification are . I will get it clarified with Proff RKK . Also the scaffolding for scanf , and using debug registers for debugging is done. The floppy thing is still hanging in the air.Also discovered a bug in the printk routine while scrolling.
Did a lot of bugfixes, though many more remain and am now trying to fix up the static data memory allocation problems I have been having , ie I am trying to fit all the global data into its correct place. Debugging this thing is really hard , I am trying to find a kernel debugger that can work on bochs to debug this . I think by this weekend the first version of my assignment should be ready .
I have done the following till now:
Wrote code for the main loop , the interrupt handler and routine, code to run and load the program (right now from memory itslf like some m/c code which I specify) . And code to create a tss , also modified by memory routines to install the area in the gdt . I will test all this (right now don't think it will even compile!) after my quizes get over on 28th.
I have decided on a design. Also wrote a couple of utility functions for memory management and keyboard scanf routines . But have yet to test out my floppy reading routines . I acknowledge the source code of Kernel Toolkit by Luis sigo and linux 0.11 by linus to help me figure out the design .
Sorry for not posting up updates before this , this is what I have been upto :