manio http://www.manio.org/blog Fri, 03 Feb 2012 04:16:30 +0000 en hourly 1 http://wordpress.org/?v=3.2.1 How to Add New System Calls to Minix 3.1.8 http://www.manio.org/blog/how-to-add-new-system-calls-to-minix-3-1-8/?utm_source=rss&utm_medium=rss&utm_campaign=how-to-add-new-system-calls-to-minix-3-1-8 http://www.manio.org/blog/how-to-add-new-system-calls-to-minix-3-1-8/#comments Fri, 03 Feb 2012 04:16:30 +0000 admin http://www.manio.org/blog/?p=879 There are some tutorials on the Internet. However, they are not up-to-date. Currently Minix has the latest version 3.1.8. It has some changes, say a virtual file system. So a  newer tutorial is necessary. This article also includes a part for adding system calls to servers that are not visible to users. For example, users can not call the functions in MFS directly because there is a middleware layer, VFS, between them. In order to call some function in MFS, one need to add a system call to MFS and then call it from VFS. The steps of adding system calls to MFS is slightly different.

I had an OSDI class last semester, which required us to do some Minix kernel modification. I learned a lot from it.

Here are the steps to add a new system call to Minix 3.1.8.

Suppose we need to add a system call, setmgroup(), to Process Manager(PM).

  1. Find an unused entry in /usr/src/include/minix/callnr.h, add a line. For example
    #define SETMGROUP 70
  2. In /usr/src/servers/pm/table.c, find the corresponding  entry (with same number) in the table and change its name to setmgroup
  3. Add
    _PROTOTYPE( int do_setmgroup, (void) );

    to /usr/src/servers/pm/proto.h

  4. Add the function body of do_setmgroup() to /usr/src/servers/pm/misc.c
  5. Go to /usr/src/servers/pm/, and
    $make

    To check if it can pass the compilation.

  6. create file mcastlib.h in /usr/include/
    add the setmgroup(int, int) to this file, which includes _syscall()
  7. cd /usr/src/tools
    make hdboot
    reboot
  8. Test it by using the system call.

The mcastlib.h is like this:

#include <lib.h>
#include <unistd.h>
#define setmgroup _setmgroup

PUBLIC int setmgroup ( int _groupid )
{
message m;
m.m1_i1 = _groupid;
return( _syscall( PM_PROC_NR, SETMGROUP, &m) );
}

Add a System Call to MFS

Suppose we need to add a system call named listblocknum() to MFS.

  1. add an entry in /usr/src/include/minix/vfsif.h
    add a line:

    #define REQ_LISTBLOCKNUM (VFS_BASE + 33)

    Add 1 to the number of request

    #define NREQS 34
  2. find the entry in the table and change its name to fs_listblocknum in /usr/src/servers/mfs/table.c
  3. find the entry in the table and change its name to no_sys in /usr/src/servers/hgfs/table.c and other file systems like ext2, etc.
    If you already changed NREQS and still got the error “array size is negative”. Try to “$make includes” in /usr/src. Then try to make in hgfs again.
  4. add
    _PROTOTYPE( int fs_listblocknum, (void) );

    to /usr/src/servers/mfs/proto.h

  5. add the function body of fs_listblocknum () to /usr/src/servers/mfs/misc.c
  6. $cd /usr/src/servers/mfs/
    $make
  7. go to /usr/src/tools,
    $make hdboot

After doing this, VFS can send message to MFS to call a function, just like USER can send a message to VFS to call a function. Refer to request.c in VFS to see how to call the functions in MFS.

 

]]> http://www.manio.org/blog/how-to-add-new-system-calls-to-minix-3-1-8/feed/ 0 Commonly used R commands (statistics) http://www.manio.org/blog/commonly-used-r-commands/?utm_source=rss&utm_medium=rss&utm_campaign=commonly-used-r-commands http://www.manio.org/blog/commonly-used-r-commands/#comments Fri, 03 Feb 2012 02:48:30 +0000 admin http://www.manio.org/blog/?p=842 When I say Ease of Use Improved, I mean you can simply copy, paste and run the codes in this post, without referring to other places, without downloading a data file and read it from R. This is how I like a blog article to be. You don’t need to read the whole article. You just need to Ctrl+F what your need and copy the codes there and run it.

I use R in Windows and sometimes Linux. The version is 2.13.0. The following scripts should be applicable to other versions.

Read a File to a Table

congold<-read.table("C:/Users/Jun/Dropbox/Research/LQCD/exp/96.24.24.24/congrad.old.txt", header=T)

Hmm.. You can’t copy and run this in your system, since you don’t have that file. congold is a table,  the first argument of read.table() is the path of the file. In Windows, you should use “/” in the path instead of “\”.

Boxplot

d = rnorm(10)
t = rep(c(1,2),c(5,5))
boxplot(d~t)

Get subset

df = data.frame(col1=c(1,2,3,4),col2=c(1,1,2,2))
subset(df,col2==2)

Find out how many unique items in a  list

a = c(5,5,6,6,6)
length(unique(a))

Viewing Several Graphs

In Windows

windows()

In Linux

X11()

In Mac

quartz()

 Delete Columns by Names

df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
df <- df[,-which(names(df) %in% c("z","t"))]

An easier way:

df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
df <- subset(df, select=-c(z,t))

Actually, it is done by selecting the columns you want. So we have the following:

Select Columns by Names

df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
df[, c("x","y")]
df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
subset(df, select=c(x,y))

Print out Column Names

df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
names(df)

Change Column Names

df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
names(df)[[1]]="newNameForColumn1"
df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
names(df)=c("newNameForColumn1", "newNameForColumn2", "newNameForColumn3","newNameForColumn4")
df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
names(df)[which(names(df)=="y")]= "NewNameOf_y"

Reduction Plot

library(lattice)
x = 1:100
y = rnorm(100)
xyplot(x~y, type=c("r","p"))

Finding out 95%th, 99%th of Each Category

library(doBy)
x = rep(c(1,2),50)
y = rnorm(100)
summaryBy(y~x, data=df, FUN=function(x){quantile(x,c(0.95,0.99))})
x = rep(c(1,2),50)
y = rnorm(100)
aggregate(y~x, data = df, function(x){quantile(x,0.95)})
aggregate(y~x, data = df, function(x){quantile(x,0.99)})

Get Median of Each Factor in a data frame (each type has many rows)

x = rep(c(1,2),50)
y = rnorm(100)
aggregate(y~x, data = df, median)

To count rows or columns

df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
nrow(df)
df <- data.frame(x=rep(1,3), y=rep(2,3), z=rep(3,3), t=rep(4,3))
ncol(df)

Create empty matrix or vector

mymatrix <- mat.or.vec(2,3)

Replace data in data frame

tmp = data.frame("a"=c(1,2,3,4))
selected = tmp == 2
selected
tmp[selected] = 22
tmp

Convert Factor to Number

size <- factor(c(55,44,33,22,11))
size
as.numeric(size)
levels(size)[size]
as.numeric(levels(size)[size])

Change the order of colums

df = data.frame("a"=c(1,1), "b"=c(2,2), "c"=c(3,3))
df
df = subset(df, select=c(c,b,a))
df

Order Data Frame

df = data.frame(a=c(4,5,6),b=c(9,8,7))

df = df[order(df$b),]
df = data.frame(a=c(4,5,6),b=c(9,8,7),c=c(11,12,12))
df
df[order(df$c,df$b),]

Too much to organize from my note…

Maybe I’ll pick it up later, nor not….

]]> http://www.manio.org/blog/commonly-used-r-commands/feed/ 0 David Ferrucci: why did IBM Watson believe Toronto was a US city? http://www.manio.org/blog/david-furracciwhy-did-ibm-watson-believe-toronto-was-a-us-city/?utm_source=rss&utm_medium=rss&utm_campaign=david-furracciwhy-did-ibm-watson-believe-toronto-was-a-us-city http://www.manio.org/blog/david-furracciwhy-did-ibm-watson-believe-toronto-was-a-us-city/#comments Sat, 27 Aug 2011 05:59:38 +0000 admin http://www.manio.org/blog/?p=834 On July 27th, Dr. David Ferrucci came to Fermi to give a talk about IBM Watson. He talked about the architecture of Watson, some technical details, how the team members cooperated with others.

He says that Watson got its knowledge from only 100GB of data. After pre-processing for fast calculation in the game, it became 800GB. It was all in the memory in the game, because disks are slow.

Knowledge was not strictly categorized. Like, cat is a kind of animal, pencil is a kind of tool. Watson did not do this. I think one of the reasons is that it is impossible to build categories for all the things in the world. It just had to be fuzzier to be general.

Regarding why Watson believes Toronto was  a US city, the explanations were:

  • Even though the question belongs to a category about US cities, the answer does not have to be a US city, according to some previous questions of Jeopardy!.
  • The confidences on Chicago and Toronto were so close. (Chicago had a chance to win.)
  • Cities are not categorized as said before. So Toronto MIGHT be a US city. They even have a team in NBA… This is the way how Waston discovers the answer: get clues, connect them.

 

 

My phone has to make loud sound when taking a picture. So I waited until everybody was laughing so it would cover the sound of my phone. Succeeded!

 

]]> http://www.manio.org/blog/david-furracciwhy-did-ibm-watson-believe-toronto-was-a-us-city/feed/ 3 Things Learned from This Summer Internship http://www.manio.org/blog/things-learned-from-this-summer-internship/?utm_source=rss&utm_medium=rss&utm_campaign=things-learned-from-this-summer-internship http://www.manio.org/blog/things-learned-from-this-summer-internship/#comments Thu, 25 Aug 2011 15:54:48 +0000 admin http://www.manio.org/blog/?p=820
  • It is worth automating the procedures you will do a lot. For example, pulling out data for analysis from output files, data analysis procedure.
  • It is worth learning the tool for a period of dedicated time. Otherwise, it would waste a lot time googling and you don’t even know some very useful knowledge/tricks to do some basic things. The basic skills you know may make you creative when using the tools.
  • Ask people, discuss with people. It tides up your mind, gives you new ideas. Never ignore those things seems trivial. They might be very scary when you rethink them as you write the final report. The whole project can be worthless and you could have avoid it by sending an email to ask somebody.
  • Never believe you can finish all the jobs in one day. Things change. People make mistakes. Machines crash. Meetings.. Car breaks.. JUST PLAN IT WELL. Give yourself spare time.
  • Never sleep without a plan for the next day.
  • Keep clear log of what you have done every day. It is a MUST DO. Since you may forget what you have done and had a conclusion. Sometimes you can’t believe some conclusions you made are true. You need to check what you’ve done and your proof. Writing log everyday is also helpful when you write the report/paper. The log needs to be well structured.
  • Work hard.
  • Keep notes for things you learn. My R notes helped a lot.

    • ]]>     http://www.manio.org/blog/things-learned-from-this-summer-internship/feed/ 4     The Tipping Point http://www.manio.org/blog/the-tipping-point/?utm_source=rss&utm_medium=rss&utm_campaign=the-tipping-point http://www.manio.org/blog/the-tipping-point/#comments Mon, 31 Jan 2011 03:54:31 +0000 admin http://manio.org/?p=752 I have no strong feelings after reading Tipping Point. Gladwell kind of does not put the examples very close to his argument in some parts. He often goes out of track and pull it back at the end of the chapters. Or I should say he does not state clearly the relation between the examples and theories. I distracted sometimes while reading.

    It is not a bad book though. It introduces several theories that may help us understanding the world: the law of the few, the stickiness factor and the power of context. It is all about how things get popular and how/why they spread.

    Here I only want to supply some supplementary interesting materials that mentioned in the book.

    The author’s website:

    http://www.gladwell.com/

    The author’s talk on TED:


    http://www.ted.com/talks/malcolm_gladwell_on_spaghetti_sauce.html

    Stanford Prison Experiment:

    http://www.prisonexp.org/

    Phil Zimbardo’s website (the supervisor of Stanford Prison Experiment) :

    http://zimbardo.socialpsychology.org/

    Phil Zimbardo’s TED talk, Philip Zimbardo shows how people become monsters … or heroes:

    http://www.ted.com/talks/philip_zimbardo_on_the_psychology_of_evil.html Phil married the girl who stopped his prison experiment:)

    Documentary of Stanford Prison Experiment:

    http://www.youtube.com/watch?v=1GpuB_Q2RUw


    http://www.youtube.com/watch?v=FHc5vX1m_zk&feature=related

    http://www.youtube.com/watch?v=kg37p1RxnSk&feature=related

    Recent movie based on Standford Prison Experiment:

    http://www.imdb.com/title/tt0997152/

    ]]>     http://www.manio.org/blog/the-tipping-point/feed/ 0     Long after Deep Blue, IBM wants to beat human again http://www.manio.org/blog/long-after-deep-blue-ibm-wants-to-beat-human-again/?utm_source=rss&utm_medium=rss&utm_campaign=long-after-deep-blue-ibm-wants-to-beat-human-again http://www.manio.org/blog/long-after-deep-blue-ibm-wants-to-beat-human-again/#comments Fri, 28 Jan 2011 02:00:57 +0000 admin http://manio.org/?p=741 On this coming Feb 14, we are expecting the human-like IBM Watson to fight in the Jeopardy! show. It is going to be an interesting and exciting Machine vs. Human show. After two years of working, IBM’s scientists are hoping this new challenge will push AI to a new height. We know how hard to learn a language. But it’s even harder to understand those slang, joke and so on, which requires experience of the culture, the history…

    IBM is so good at merging their research with marketing campaign. I missed the Deep Blue last time. This time, I am gonna watch it live.

    Report from MIT Technology Review

    IBM’s introduction

    Watson project

    Jeopardy! website

    A tiny simulation to help you understand the game

    ]]>     http://www.manio.org/blog/long-after-deep-blue-ibm-wants-to-beat-human-again/feed/ 0     SESC Tutorial – TODO List http://www.manio.org/blog/sesc-tutorial-todo-list/?utm_source=rss&utm_medium=rss&utm_campaign=sesc-tutorial-todo-list http://www.manio.org/blog/sesc-tutorial-todo-list/#comments Fri, 11 Jun 2010 04:39:15 +0000 admin http://manio.org/?p=674 This is a TODO list, listing what should be done next for this tutorial, to help researchers understand and modify SESC.

    • How to run a therm/temperature test step by step
      Any contributors?
    • How to run a benchmark
    • How to modify the simulater configuration (.conf) Started. By Jun He(jhe24[at]iit.edu).
    • Illustration of the key classes. The illustration should show the main functions of the classes (I mean why these classes exsit.) and relationships among them. So the readers can get the a clear picture of SESC. Any contributors?
    • How to modify the cache replace policy. We believe a good example worth thousands of words. This cache task is simple, without too many relations to other objects. So it might be the best simple example to demostrate hacking. Started. By  Jun He(jhe24[at]iit.edu), Yao Wang(Yao.Wang[at]tudelft.nl).
    • Set up a wiki?
    • Even more…
    ]]>     http://www.manio.org/blog/sesc-tutorial-todo-list/feed/ 3     SESC Tutorial – 3. Prepare for Hacking http://www.manio.org/blog/sesc-tutorial-3-prepare-for-hacking/?utm_source=rss&utm_medium=rss&utm_campaign=sesc-tutorial-3-prepare-for-hacking http://www.manio.org/blog/sesc-tutorial-3-prepare-for-hacking/#comments Tue, 08 Jun 2010 09:26:21 +0000 admin http://manio.org/?p=621 http://manio.org/blog/category/computer-science/sesc-tutorial

    Written by Jun He (jhe24(at)iit.edu), June 2010.

    This chapter introduces you some tips to understand SESC. If you want to modify SESC for your research, this might be a good start.

    Index

    1. Resources for learning SESC
    2. How to Learn SESC?
    3. Pre-knowledge for Hacking SESC
    4. How to Understand SESC and Start Hacking
    5. The Illustration of SESC Source Code Tree (mind map)
    6. How to configure a heterogeneous MPSoC

    Resources for learning SESC

    How to Learn SESC?

    1. Add http://sesc.sourceforge.net to your bookmark
    2. Subscribe to sesc-news@lists.sourceforge.net
    3. Read all the files in doc directory
    4. See the ./src/main.cpp example
    5. Understand the callbacks (DInst.h is easy)
    6. Understand the configuration files (*.conf)

    From slide1.pdf If someone wants to read the source codes of SESC, I think he/she should start from src/libcore/main.cpp Another tutorial is in the README.hacker in the src/docs. It’s very important for SESC hackers.

    Pre-knowledge for Hacking SESC

    Get a thick book on computer architecture on hand. Wikipedia and Google every unkown term you see.

    How to Understand SESC and Start Hacking

    The booting of SESC/the main() of SESC. This is a wonderful demonstration of SESC or any simulator system. The whole system is OOPed. Every part in a real architecture is represented as an object in C++. If we want to change something, just change the according object.

    int main(int argc, char **argv, char **envp) {
    DummyMemorySystem *cm[NPROC];
    MyProc *core[NPROC];
    osSim = new OSSim(argc, argv, envp);
    for(Pid_t i = 0; i > NPROC; i ++) {
        cm[i] = new DummyMemorySystem(i);
        core[i] = new MyProc(cm[i], i);
    }
    osSim->boot();
    for(int i = 0; i < NPROC; i ++) {
        delete core[i];
        delete cm[i];
    }
    delete osSim;
}

    from slide1.pdf The real code is in the src/libcore/main.cpp

    The Illustration of SESC Source Code Tree (mind map)

    How to configure a heterogeneous MPSoC

    refer to https://lists.soe.ucsc.edu/pipermail/sesc/2007-February/000175.html Also refer to a UCLA student’s research progress using sesc: http://www.cs.ucla.edu/honors/UPLOADS/2006-07/salzman/progress.txt. It might give you some hints.

    To be continued…

    ]]>     http://www.manio.org/blog/sesc-tutorial-3-prepare-for-hacking/feed/ 14     SESC Tutorial – 2. Run HelloWorld http://www.manio.org/blog/sesc-tutorial-2-run-helloworld/?utm_source=rss&utm_medium=rss&utm_campaign=sesc-tutorial-2-run-helloworld http://www.manio.org/blog/sesc-tutorial-2-run-helloworld/#comments Tue, 08 Jun 2010 05:40:35 +0000 admin http://manio.org/?p=600 http://manio.org/blog/category/computer-science/sesc-tutorial

    Written by Jun He (jhe24(at)iit.edu), June 2010.

    In this chapter, we compile and run the first program on SESC. After that, we can get a report from SESC and check the performance.

    Index

    1. How to Setup Building Environment
    2. How to compile a program and run it on SESC
    3. How to run multi-thread program
    4. How to visualize the report file

    How to Setup Building Environment

      1. $vim setupSescEnvironment
    set +h

export PATH=$HOME/sescutils/install/bin:$HOME/sesc-build:$PATH

echo "Setup Finished!"
    1. $source setupSescEnvironment

    How to compile a program and run it on SESC

      1. $vim hellosesc.c
    #include <stdio.h>int main()
    {
    printf(“hello sescn”);
    return 0;
    }
      1. Compile hellosesc.c by mips gcc
        $mipseb-linux-gcc hellosesc.c

    ERROR

    /home/manio/sescutils/install/lib/gcc/mipseb-linux/3.4.4/../../../../mipseb-linux/bin/ld: /home/manio/sescutils/install/lib/gcc/mipseb-linux/3.4.4/crtend.o: warning: linking PIC files with non-PIC files/home/manio/sescutils/install/lib/gcc/mipseb-linux/3.4.4/../../../../mipseb-linux/bin/ld: /home/manio/sescutils/install/lib/gcc/mipseb-linux/3.4.4/../../../../mipseb-linux/lib/crtn.o: warning: linking PIC files with non-PIC files

    Note: This is because you do not compile the .c file with the same options of glibc. So refer to the build options of glibc (build-3-glibc) and set the gcc options as follows:

      1. $mipseb-linux-gcc -O2 -mips2 -mabi=32 -fno-PIC -mno-abicalls hellosesc.c
      2. Success built, BUT THE OUTPUT EXECUTABLE CANNOT BE RUN IN SESC!
    manio@jun-desktop:~/sescworkdir$ sesc.tst -h0x800000 -c../esesc/confs/mem.conf ./a.out <../esesc/tests/tt.inERROR: .ctors data sections not contiguousSegmentation fault

    The following references can help you understand this problem.

    Reference 1:

    https://lists.soe.ucsc.edu/pipermail/sesc/2008-May/000466.html

    When you compile you need to use “-static -Wa,-non_shared”
    Also, when you link, make sure that all the sections are continues.
    You can do it
    adding this option to the linking process.
    -Wl,–script=$(XTOOLSPREFIX)/mipseb-linux/lib/ldscripts/mint.x,-static

    Reference 2

    https://lists.soe.ucsc.edu/pipermail/sesc/2007-October/000329.html

    I am not sure about the reason for the problem. My guess is that
    SESC/MINT interface
    requires continuous data sections (.bss,.data.text….). Your program
    may have very large
    static data allocation and gcc may decide not to have continuous
    sections.
    If you use the “mipseb-linux-readelf -S foo” command where foo is
    your binary, it will
    tell you the sections.
    If you look at the mint.x file from gcc, it shows how to change it.
    Maybe your program
    uses some section not addressed on mint.x.

    Reference 3

    https://lists.soe.ucsc.edu/pipermail/sesc/2007-October/000328.html

    /usr/local/bin/mipseb-linux-g++ -mips2 -mno-abicalls -mabi=32 -mtune=r6000
    -msplit-addresses -I. -Wa,-non_shared -I.
    -I/usr/local/mipseb-linux/include/c++/3.2
    -I/usr/local/mipseb-linux/include/c++/3.2/backward
    -I/usr/local/mipseb-linux/include/c++/3.2/mips-linux-gnu -static SOURCE
    FILES HERE -o test -Wl,–script=/usr/local/mipseb-linux/lib/ldscripts/mint.x
      1. SOLUTION

    mint.x should be used to link the objects.

    manio@jun-desktop:~/sescworkdir$ mipseb-linux-gcc -mips2 -mabi=32 -static -Wa,-non_shared -mno-abicalls -Wl,–script=/home/manio/sescutils/install/mipseb-linux/lib/ldscripts/mint.x,-static hellosesc.c -o hellosesc.6manio@jun-desktop:~/sescworkdir$

    In this solution, gcc passes some options to the linker(ld) by -Wl.

    -Wl,option

    Pass option as an option to the linker.  If option contains commas, it is split into multiple options at the commas.

    –script=scriptfile

    Use  scriptfile as the linker script.  This script replaces ld’s default linker script (rather than adding            to it), so commandfile must specify everything necessary to describe the  output  file.     If  scriptfile does not exist in the current directory, “ld” looks for it in the directories specified by any preceding -L options.  Multiple -T options accumulate.

    -static

    Do not link against shared libraries.  This is only meaningful on platforms for which shared libraries are supported.  The different variants of this option are for compatibility with various systems.  You may use this  option  multiple times on the command line: it affects library searching for -l options which follow it. This option also implies –unresolved-symbols=report-all.  This option  can  be  used  with  -shared. Doing  so  means  that a shared library is being created but that all of the library’s external references must be resolved by pulling in entries from static libraries.

    How to run multi-thread program

      1. $cd $HOME/esesc/src/libapp
      2. add #include <stdint.h> to sescapi.h
      3. create object of sesc_thread.c
    manio@jun-desktop:~/esesc/src/libapp$ mipseb-linux-gcc -g -mips2 -mabi=32 -Wa,-non_shared -mno-abicalls -c -o sesc_thread.o sesc_thread.c
      1. build executable. Please note: we should use mint.x, which is the elf for sesc mips.
    manio@jun-desktop:~/esesc/src/libapp$ mipseb-linux-gcc -o hello.sesc hello.c -g -mips2 -mabi=32 -Wa,-non_shared -mno-abicalls sesc_thread.o -static -Wl,–script=/home/manio/sescutils/install/mipseb-linux/lib/ldscripts/mint.x,-static
      1. $cd ~/sesc-build
      2. run the simulator
    manio@jun-desktop:~/sesc-build$ ./sesc.smp -c ../esesc/confs/smp.conf.hello.multithread ../esesc/src/libapp/hello.sesc
      1. show the report
    manio@jun-desktop:~/sesc-build$ ../esesc/scripts/report.pl -last

    References:

    https://lists.soe.ucsc.edu/pipermail/sesc/2007-October/000308.html

    http://ce.et.tudelft.nl/~pepijn/doc/sesc.html

    How to visualize the report file

    Each line of the report file is consisted of:

    Object which generated the data: field1 = value1: field2 = value2

    manio@jun-desktop:~/sesc-build$ ../esesc/scripts/report.pl sesc_hellosesc.6.uJN6k3
    ]]>     http://www.manio.org/blog/sesc-tutorial-2-run-helloworld/feed/ 15     SESC Tutorial – 1. Install Whole SESC Step by Step http://www.manio.org/blog/sesc-tutorial-1-instaall-whole-sesc-step-by-step/?utm_source=rss&utm_medium=rss&utm_campaign=sesc-tutorial-1-instaall-whole-sesc-step-by-step http://www.manio.org/blog/sesc-tutorial-1-instaall-whole-sesc-step-by-step/#comments Mon, 07 Jun 2010 16:45:48 +0000 admin http://manio.org/?p=577 http://manio.org/blog/category/computer-science/sesc-tutorial

    Written by Jun He (jhe24(at)iit.edu), June 2010.

    SESC is good. But after so many years, there are still few documents.  Researchers are often dragged by some tiny troubles. The research can really start, after they can run the hello world. This tutorial aims to reduce the time spending on setuping the evnironment. It lets the researchers pass the tedious evironment setuping process and get down to the exciting researches as soon as possible.

    This tutorial is written by Jun He. His e-mail is jhe24(AT)iit.edu. His homepage is http://manio.org. There may be some mistakes in this tutorial. Comments are appreciated. Also, I’ll be very glad if someone wants to work on this tutorial with me.

    In this tutorial, I also list the errors which might encounter, so the readers can find out what to do when they do something wrong.  The errors make the reader know more about SESC.

    Index

    1. What’s SESC?
    2. My Working Environment
    3. How to Install Old GCC3.4 by deb package in Ubuntu 8.04
    4. How to build SESC utils
    5. How to Build SESC source code

    What’s SESC ?

    Homepage: http://sesc.sourceforge.net/index.html

    SESC is a cycle accurate architectural simulator. It models a very wide set of architectures: single processors, CMPs, PIMs, and thread level speculation.

    SESC started as the pet project of Jose Renau while doing his PhD at Urbana-Champaign in the IACOMA group. Currently, he is a new faculty at University of California, Santa Cruz.

    My Working Environment:

    • Ubuntu 8.04 Desktop LTS (Mine is on VMWare 7.01) Accuatlly, you can run sesc in any Linux distributions which are not too old.
    • SESC: checked out in 05/25/2010 by cvs.
    • Internet Access. If you can access internet in Linux, the process of installing will be much easier, because you can use apt-get install.
    • gcc 3.4.x. Usually, this version of gcc is not installed in your OS. The way of installing this version of gcc will be introduced immediately.

    Please follow the following howtos one by one to setup SESC.

    How to Install Old GCC3.4 by deb package in Ubuntu 8.04

    Why we need a older version of gcc?

    Because the new version will cause errors, weird errors. The reason for this might be that some new features have been added to gcc 4.x, and they are not compatible with older codes. SESC were mainly developed around 2005, using gcc 3.4. If we use gcc 3.4, we’ll get the same SESC as Jose, the father of SESC.

      1. Download the following 3 deb packages from http://archive.ubuntu.com/ubuntu/pool/main/g/gcc-3.4/
        a)         cpp-3.4_3.4.4-6ubuntu8_i386.deb (1707096 bytes
        b)        gcc-3.4_3.4.4-6ubuntu8_i386.deb (484408 bytes)
        c)        gcc-3.4-base_3.4.4-6ubuntu8_i386.deb (163028 bytes)

    Note: the version number in the online archive may not match the ones above, but it’s ok. Please download only the three packages listed above, no more no less. Otherwise, you will encounter some errors.

      1. Follow the instructions in http://ubuntuforums.org/showthread.php?t=79896, then you can have GCC.installed.
      2. Now, if you want to install g++, you have to install them by dpkg. Because if you use GUI-double-click to install g++-3.4_3.4.6-1ubuntu2_i386, ubuntu says “libstdc++6-dev_3.4.6-1ubuntu2_i386 is needed”. But if you double click libstdc++6-dev_3.4.6-1ubuntu2_i386 first, ubuntu says “g++-3.4_3.4.6-1ubuntu2_i386 is needed”.
      3. Download the tow g++ deb packages mentioned in step 3 from http://archive.ubuntu.com/ubuntu/pool/main/g/gcc-3.4/
      4. $sudo dpkg -i g++-3.4_3.4.6-1ubuntu2_i386.deb libstdc++6-dev_3.4.6-1ubuntu2_i386.deb

    The following steps are from http://blog.sina.com.cn/s/blog_48a44f390100igad.html

    1. After the steps above, gcc3.4 is installed in your system. So now you have two versions of gcc in you system. The default is gcc 4.4, we should change it, in order to compile source codes of SESC by gcc 3.4
    2. List the all gcc you have in your system by:
      $ls /usr/bin/gcc* -l
    3. Add options for gcc 3.4 and gcc 4.4. The version number might not match yours. If not, change yours to the correct ones.
      $ sudo update-alternatives –install /usr/bin/gcc gcc /usr/bin/gcc-4.2 40
      $ sudo update-alternatives –install /usr/bin/gcc gcc /usr/bin/gcc-3.4 30
    4. Alternate to gcc 3.4 
      manio@jun-desktop:~/esesc/src/libsuc$ sudo update-alternatives --config gcc
[sudo] password for manio:
There are 2 alternatives which provide `gcc'.
Selection    Alternative
-----------------------------------------------
+        1    /usr/bin/gcc-4.2
*         2    /usr/bin/gcc-3.4
Press enter to keep the default[*], or type selection number:
    5. Enter 2, then gcc 3.4 will be chosen.
    6. Now gcc 3.4 is successfully installed in your system. From now on, you should use the gcc 3.4 to compile SESC source code.

    How to build SESC utils

    What’s sescutils?

    Sescutils is a group of tools for building programs to run on SESC. These tools include gcc, gdb, glibc, etc.

    Why do we need utils in SESC?

    Actually, sescutils is a cross-compile toolchain, which is used to compile programs on PC for running on MIPS. SESC is a MIPS architecture simulator.

      1. Download the sescutils from https://sourceforge.net/project/showfiles.php?group_id=49065
      2. move the package to your $HOME. My $HOME=/home/manio. In the following parts of this tutorial, I will use /home/manio directly. You should
      3. $cd $HOME
      4. $tar jxvf sescutils.tar.bz2
      5. make /bin/sh link to /bin/bash by:
    sudo rm /bin/shsudo ln -s /bin/bash /bin/sh

    Ubuntu uses dash, not bash. But the shell scripts in SESC are written by bash. So we have to change to bash.

      1. modify file build-common in /home/manio/sescutils/build-mipseb-linux
        GNUSRC=$HOME/sescutils/src
        PREFIX=$HOME/sescutils/install
        Note: GNUSRC is the position of the source codes. PREFIX is the position where the sescutils binary files will be installed to.
      2. $cd $HOME/sescutils/build-mipseb-linux
      3. $./build-1-binutils.
        error
    gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -g -O2 -o ar arparse.o arlex.o ar.o not-ranlib.o arsup.o rename.o binemul.o emul_vanilla.o bucomm.o version.o filemode.o  ../bfd/.libs/libbfd.a ../libiberty/libiberty.a ./../intl/libintl.aarlex.o: In function `main’:

    /home/manio/sescutils/build-mipseb-linux/obj/binutils-build/binutils/arlex.c:1: multiple definition of `main’

    arparse.o:/home/manio/sescutils/build-mipseb-linux/obj/binutils-build/binutils/arparse.c:1: first defined here

    ar.o: In function `main’:

    /home/manio/sescutils/src/binutils/binutils/ar.c:342: multiple definition of `main’

    arparse.o:/home/manio/sescutils/build-mipseb-linux/obj/binutils-build/binutils/arparse.c:1: first defined here

    bucomm.o: In function `make_tempname’:

    /home/manio/sescutils/src/binutils/binutils/bucomm.c:425: warning: the use of `mktemp’ is dangerous, better use `mkstemp’ or `mkdtemp’

    ar.o: In function `mri_emul’:

    Note: These errors are caused by the missing of bison and flex, which are used to parse the codes. So, we install them in the following steps. Before installing, make sure you can access the internet.

      1. Install bison
        $sudo apt-get install bison
      2. $./build-1-binutils. Same error with 3
      3. $sudo apt-get install flex
      4. ./build-1-binutils
        Success
      5. $./build-2-gcc-core
        Note: if you use gcc 4.x to compile sescutils, you will meet this error. To solve this problem, change to gcc3.4 by following the How to Install old gcc3.4 by deb in Ubuntu 8.04.
        If you still can not get rid of the error, refer to the comments of this page. Your bison or flex should be the correct version.
    gcc -c   -g -O2 -DIN_GCC -DCROSS_COMPILE  -W -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -pedantic -Wno-long-long  -Wno-error  -DHAVE_CONFIG_H    -I. -I. -I/home/manio/sescutils/src/gcc-3.4/gcc/gcc -I/home/manio/sescutils/src/gcc-3.4/gcc/gcc/. -I/home/manio/sescutils/src/gcc-3.4/gcc/gcc/../include  c-parse.c -o c-parse.ogcc: c-parse.c: No such file or directory

    gcc: no input files

    make[1]: *** [c-parse.o] Error 1

    make[1]: Leaving directory `/home/manio/sescutils/build-mipseb-linux/obj/gcc-core-build/gcc’

    make: *** [all-gcc] Error 2
      1. $./build-3-glibc
    manio@jun-desktop:~/sescutils/build-mipseb-linux$ ./build-3-glibcchecking build system type… Invalid configuration `unknown-linux’: machine `unknown’ not recognized

    configure: error: /bin/sh /home/manio/sescutils/src/glibc-2.3.2/scripts/config.sub unknown-linux failed

    make: *** No rule to make target `all’.  Stop.

    Note: This error is caused by the wrong value of BUILD in build-common file. The uname –p command cannot output the right version of the building system, i.e. your PC.
    To correct this problem, simply change the value of BUILD to “i686-pc-linux-gnu”. By the way, there is another value, host, which is the platform in which you want to run your compiled glibc. The target system is the platform in which you will run your program with glibc.
    The typical toolchain is: build=your pc, host=your pc, target=arm/mips…

      1. $./build-3-glibc
        Success.
      2. $./build-4-gcc
        Success
      3. $./build-5-gdb

    ERROR

    checking for wctype… yeschecking for library containing gethostbyname… none required

    checking for library containing socketpair… none required

    checking for library containing waddstr… no

    checking for library containing tgetent… no

    configure: error: no termcap library found

    make: *** [configure-gdb] Error 1

    Note: A lib is missing. Install it by the following steps.

    1. $ sudo apt-get install libncurses5-dev
    2. $./build-5-gdb
      Success.
    3. Now, the sescutils is completely built. You can use it to build programs for SESC later.

    How to Build SESC source code

      1. Install CVS
        $sudo apt-get install cvs
      2. Download SESC source code by CVS
        $cvs -d:pserver:anonymous@sesc.cvs.sourceforge.net:/cvsroot/sesc login
        Note: just press Enter when password is requested.
        $cvs -z3 -d:pserver:anonymous@sesc.cvs.sourceforge.net:/cvsroot/sesc co -P sesc
      3. Move the source code to $HOME/esesc
      4. Read the $HOME/esesc/REAMME. It will give you some steps to install SESC.
      5. $cd ~
      6. $mkdir sesc-build
      7. $cd sesc-build
      8. $../esesc/configure
        Note: if you download SESC in Windows, you will meet the following error.
        ERROR: elif unexpected…
        REASON: cannot use DOS text format in Linux
        SOLVE: open configure by vim, :set ff=unix
      9. $sudo apt-get install binutils
        Note: this binutils is for building SESC source code. The binutils in sescutils is for building programs running on SESC.
      10. $make
        ERROR: USHRT_CHAR undefined
        SOLUTION: include limits.h file in esesc/src/libcore/FetchEngine.cpp file. Add
        #include <limits.h>
      11. $make
        ERROR:
        /home/manio/SESC/build/../esesc/src/libmint/subst.cpp:52:26: error: linux/dirent.h: No such file or directory
        SOLUTION:
        - Do not include linux/dirent.h, use dirent.h instead
      12. $make
        ERROR‘uint32_t’ was not declared in this scope
        SOLUTION: add #include <stdint.h> wherever you see this error.
        Reference:
        http://www1.cs.columbia.edu/~youngjin/discus/messages/54/59.html?1259617974
        Hi Alejandro,        It maybe caused by the strictness of the C compiler provided in Ubuntu 9.10.
        (I had almost the same thing with 9.10.)        As you mentioned, please try to include the
        following statement anytime you have the above-mentioned error:
        #include <stdint.h>
      13. $make
        ERROR: “/usr/bin/ld: cannot find -lz”
        SOLUTION: sudo apt-get install zlib1g-dev
      14. SUCCESS
      15. Test installation:
        $make testsim

    Output

    manio@ubuntu:~/SESC/build$ make testsimmake[1]: `sesc.mem’ is up to date.

    Generating sesc.conf from: /home/manio/SESC/build/../esesc/confs/mem.conf

    cp /home/manio/SESC/build/../esesc/confs/mem.conf sesc.conf

    cp /home/manio/SESC/build/../esesc/confs/shared.conf .

    ./sesc.mem -h0x800000 -csesc.conf /home/manio/SESC/build/../esesc/tests/crafty < /home/manio/SESC/build/../esesc/tests/tt.in

    static[0x1008db40-0x101b3dd4] heap[0x101b4000-0x109b4000] stack[0x109b4000-0x111ac000] -> [0x41000000-0x4211e4c0]

    Crafty v14.3

    sesc_simulation_mark 0 (simulated) @30176641

    White(1): sesc_simulation_mark 1 (simulated) @30225840

    White(1): pondering disabled.

    sesc_simulation_mark 2 (simulated) @30299339

    White(1): noise level set to 0.

    sesc_simulation_mark 3 (simulated) @30321649

    White(1): search time set to 99999.00.

    sesc_simulation_mark 4 (simulated) @30423769

    White(1): verbosity set to 5.

    sesc_simulation_mark 5 (simulated) @30449198

    White(1): sesc_simulation_mark 6 (simulated) @30751014

    White(1): search depth set to 2.

    sesc_simulation_mark 7 (simulated) @30767155

    White(1):

    clearing hash tables

    depth   time  score   variation (1)

    1   ###.##  -0.67   axb5 c6xb5

    1   ###.##  -0.08   a4a5

    sesc_simulation_mark 8 (simulated) @33091197

    1-> ###.##  -0.08   a4a5

    2   ###.##     –   a4a5

    2   ###.##  -0.65   a4a5 f6f5

    2   ###.##  -0.58   axb5 c6xb5 Ne4c5

    2   ###.##  -0.46   Rf1c1 f6f5

    2   ###.##  -0.43   Ra1c1 f6f5

    sesc_simulation_mark 9 (simulated) @37838384

    2-> ###.##  -0.43   Ra1c1 f6f5

    time:###  cpu:###  mat:-1  n:833  nps: ####

    ext-> checks:18 recaps:4 pawns:0 1rep:6

    predicted:0  nodes:833  evals:372

    endgame tablebase-> probes done: 0  successful: 0

    hashing-> trans/ref:17%  pawn:83%  used:w0% b0%

    White(1): Ra1c1

    time used: ###.##

    sesc_simulation_mark 10 (simulated) @38879583

    Black(1): execution complete.

    Here is another howto for installing SESC, by Girish Venkatasubramanian.

    ]]>     http://www.manio.org/blog/sesc-tutorial-1-instaall-whole-sesc-step-by-step/feed/ 32