Lab 02 - Preparing for Project 1 Latex, make, xmgrace and bash scripts This is most of the material from Quiz 2 on D2L Xmgrace Q1. Plots are often essential in science communication, and there are many software tools available for making plots.  Python has several packages, like matplotlib, that are versatile and have a lot of functionality.  A lightweight program for producing plots quickly from the command line is Grace.  Grace stores data and plot settings in a text file, and having the data and plotting info in one file is useful of archiving. In the Project1 directory, you should see the file called "diff_results.agr".  Enter xmgrace diff_results.agr to make the plot appear.  When this works for you, keep the plot open for the next question and answer True. Q2. The plot resulting from diff_results.agr has a number of features that you can change.  Some can be set from the command line.  Otherwise, various panels can pop up through the various menus or by double clicking.  Do the following tasks. a. Double click just outside the frame near the x-axis (or select menu Plot | Axis properties and select the x-axis to edit).  A panel should appear. b. Instead of "Step Size" as the axis label, change it to "h". Then choose the "Axis label & bar" tab and change "Char size" to 180 (you can use the arrow keys one the slider is selected).  Then choose the "Tick labels" tab and change "Char size" to 150".  Click Apply at the bottom of the panel. c. Continuing with the same panel, select the y-axis to edit, similarly change the axis label and tick label sizes to 180 and 150, respectively.  Change the axis label string to the Greek letter epsilon with a subscript by entering \xe\0\sabs as the label string. Click Apply at the bottom of the panel to see the changes. Note: The "\x" changes the font to "Symbols", the "\0" changes the font back to the default, and "\s" change the font to a subscript.  Other modes include "\S" for a superscript and "\N" to go back to normal font. If an axis label goes off the page, you can change the size of the graph by double clicking a corner of the frame, then moving the mouse, and finally clicking when you like the size. d. Save changes through the file menu File | Save. e. You can save the parameters that affect how your graph looks like through the file menu Plot | Save parameters.  In the bottom "Selection" space of the panel that opens up, type in "f1.par" at the end of the path.  Click "OK".  If you click "OK" again, it'll ask if you want to overwrite the file.  You can say yes.  Click "Cancel". f. Now create an eps file of your plot through the menu "File | Print setup".  For "Device", choose EPS.  Click "Accept".  Now choose the menu "File | Print".  It'll ask you if you want to overwrite the existing eps file.  Click "OK" g. Through the terminal, you should see that the files diff_results.agr and diff_results.eps have been updated in the PROJECT1 directory.  You should also see the f1.par file. h.  Close diff_results.agr (File | Exit or close the window). Once everything above is working, please choose "True" to answer this question. Q3 Link for string typesetting in xmgrace https://plasma-gate.weizmann.ac.il/Grace/doc/UsersGuide.html#ss7.1 Q4 Fitting.  Compile the code differentiate.f90 with gfortran differentiate.f90 (or gcc differentiate.c is using the c codes). Run the code with ./a.out The file "diff_results.dat" should now be present. It contains 3 columns, steps size (col 1), and error from forward (col 2) and centred difference schemes (col 3). Enter xmgrace -log xy -nxy diff_results.dat "-log xy" plots with both axes scaled logarithmically "-nxy" plots each column 2, 3, 4... each vs column 1. By default, xmgrace file1 file2 ... will plot column 2 vs column 1 for each file given. The resulting plot should look a little like diff_results.agr, but not quite as nice.  Close the window.  If it asks "Exist losing unsaved changes", choose "OK". Enter xmgrace -log xy -block diff_results.dat -bxy 1:3 This will plot column 3 vs column 1.  Close the window. Enter xmgrace -log xy -nxy diff_results.dat -p f1.par This will recreate the plot in diff_results.agr.  Choose menu "File | Save as", and add the file name "diff_results_fit.agr" to the path in the bottom text window and click "OK". There are a bunch of buttons on the left side of the window. There is one that has what looks like a magnifying glass on it.  Click on it.  Now select an area that includes the points on the red curve (centred difference) that look like a power law, i.e. a straight line on this double logarithmic plot, let's say the five right-most points.  You will need to click once to start the selection, and once to finish. If you mess up, press the button next to the one with the magnifying glass.  It looks like it has "AS" (all scale?) on it. If you need to move the legend, press Ctrl-L.  Then you can click on the legend and move it.  Press ESC after you are done moving the legend. Once you have only the 5 rightmost point of the red curve visible (it's okay if points from the black curve are present), select menu "Data | Transformations | Regression ". In the Regression panel, select set "S1".  This is the second, red set. Set "Type of fit" to "Power". Set "Load" to "Function". Set "Restrictions" to "Inside graph" Set "Start load at" to "1e-6", "Stop load at" to "1", and "# of points" to "10". Click "Accept".  A line should appear that goes though the 5 rightmost points of the red curve. If you mess up and generate lines that you don't want, you can double-click on the graph, click on the offending set, right-click-and-hold on that set, and then select "kill". Use the menu "Plot | Load parameters" to get a panel hat allows you to read in the f1.par file.  Once you select f1.par from the Files list, click "OK".  The graph will now look like diff_results.agr, except will include a fit to the centred difference data.  Save your work ("File | Save"). When you do the fit, a window should pop up that gives information on the fit. If you accidentally close this window, you can bring it back from the menu "Window | Console".  From the information given (you may not to enlarge the window and/or scroll), you should see that the exponent in the fit is something like 1.99994 +/- 4e-5 (0.000037).  To save this output, under the File menu of the console, select Save, and pick a file name, perhaps "f1.fit", and click "Apply" or "Accept". From the terminal, you should see that f1.fit, a text file, now exists. Double click near the bottom point of this line.  The "Set Appearance" window should appear.  Make sure the set corresponding to the fit is selected (should be G0.S2 - graph 0, set 2). Enter "Fit y = Ax\SB" (without the quotes) in the Legend String text box.  Increase the width of the line to 2.  Click "Apply".  When you are happy with the changes, press "Accept" or "Close". Save your work ("File | Save"). Exit. When you've done all of the above steps successfully, answer True for this question. Makefiles Q5 The PROJECT1 directory contains the file Makefile, which is used by the utility "make" to automate tasks with multiple file dependencies, as in compiling a program with several source files.  The general structure of a Makefile rule is target: dependency1 dependency2 ...         action1         action2         ... The target is the name of a task, often the name of a file, like an executable, that needs to be created by processing other files. After the target name, there is a colon.  After the colon is a list of files on which completing the task depends. The commands that follow are actions that need to be taken to carry out the target task.  Lines that specify actions/command must begin with a TAB (not spaces). If target is a filename, make will check to see if dependency1 or dependency2 have been updated, i.e. their timestamps are more recent than target's, or if the files on which dependency1 and dependency2 depend have been updated.  If nothing has been updated, then the actions will not be executed. Indicate all statements below that are true. Q6 By default, entering make will search for a file called Makefile.  Makefiles can make use of variables and comment lines. The contents for the PROJECT1 Makefile are # Makefile CC=gfortran TARGET2=integrate all: differentiate ${TARGET2} differentiate: differentiate.o ${CC} differentiate.o -o differentiate differentiate.o: differentiate.f90 ${CC} -c differentiate.f90 ${TARGET2}: ${TARGET2}.o ${CC} ${TARGET2}.o -o ${TARGET2} ${TARGET2}.o: ${TARGET2}.f90 ${CC} -c ${TARGET2}.f90 clean: rm -f differentiate rm -f ${TARGET2} rm -f *~ rm -f *.o By entering make differentiate make will search for the target "differentiate".  It sees that differentiate depends on differentiate.o.  It will then look for a target called "differentiate.o', and then see that it exists and depends on differentiate.f90.  Let's say we've just made some changes to differentiate.f90 (you can simluate this be entering "touch differentiate.f90".  make will determine that "differentiate" needs to be updated and so will carry out all required actions.  Actually enter the above command to see what happens.  Enter is again to see what doesn't happen. The target "all" is special for two reasons.  One, it's first and therefore the target that runs if make is called with no arguments.  Two, there are no actions, just a couple of targets that need to be checked.  With this setup, entering make will compile all code that has been updated. The target "clean" is special because it has no dependencies, nor is it a dependency for any other target.  It is called a phony target.  It is still useful.  Entering make clean will remove all object files, executables (differentiate and integrate) and funny files that being with the ~ symbol (created by certain editors or other programs when they automatically save work). Indicate all statements that are true.  Assume that you have just run make clean Q7 For projects, all code must be compiled through use of a Makefile. However, figures graphics files can also be generated with make files. In the PROJECT1 Makefile, edit the "all" line so that it reads all:    differentiate integrate diff_results_fit.eps Next, add the following rule to the Makefile. diff_results_fit.eps: diff_results_fit.agr     xmgrace -printfile diff_results_fit.eps -hdevice EPS -hardcopy diff_results_fit.agr     epstopdf diff_results_fit.eps Don't forget about using the TAB key for the commands. The first action uses xmgrace's command line capability to create an output file without any fuss.  The second creates a pdf from the eps file in case you want to use it. When you've got the above working, answer True. Latex Q8 Latex is a typesetting program that's really good for math, and many scientific publications are prepared using it.  Whereas something like Word displays the results of typesetting as you go, latex files need to be processed to produce a file that can subsequently be viewed.  Latex takes care of things like numbering figures, tables and references. It also makes use of commands that allow special formatting. In the PROJECT1 directory there should be a file called project1.tex.  Enter latex project1.tex There should be a bunch of text dumped to the screen.  It will likely tell you that it needs to be rerun because of "cross-references", so run "latex project1.tex" again. A file project1.dvi should now be in your directory.  It is a "device independent" file, and can be viewed with xdvi project1.dvi & (recall that the "&" simply starts the program running in the background so that the command line remains available). You should see the project write-up appear in a new window.  (On my macos system, for some reason, the graph doesn't appear.  Don't worry if this is the case for you on your Mac.) Edit the file project1.tex, with an editor of your choice, like nano project1.tex Near the top, you should see a line that reads, \author{Your Friendly Neighbourhood Professors} The \author{} command places the authors in a spacial place on the page with particular formatting.  Edit this line so that your name appears inside the curly brackets.  Save and exit, and then run "latex project1.tex" again. The change should be visible in the xdvi window. To convert project1.dvi into a pdf, enter dvipdfm project1.dvi A file called project1.pdf should now be in your directory.  You will need to submit a pdf version of your write-up on D2L in addition to submitting your project directory on alfven. You can also try creating a pdf directly by entering pdflatex project1.tex A more user-friendly way of working with Latex is to use a Latex editor like gummi (Ubuntu) or TeXShop (macos). Enter gummi project1.tex The GUI allows you to "compile" the document and to save/export it as a pdf. (On mac, entering open project1.tex should open project1.tex with TexShop.   If not, open it through a Finder window with TexShop.  If  you need to install TexShop, see the "Setting up at home" link on the course website.)  Using gummi, TeXShop, or other editor if those aren't working yet, change the line in project1.tex that reads \includegraphics[width=\textwidth]{diff_results} to \includegraphics[width=\textwidth]{diff_results_fit} and recompile.  The new figure you made earlier with the fit should now appear in the writeup. Select all statements below that are true. Q9 We have seen several useful command line utilities.  We shall soon see how commands can be strung together with "pipes" for greater functionality. You can place commands one after the other in a file, make the file executable, and you have the simplest sort of script.  With added functionality provided by variables, for loops, if statements, etc. you have the makings of a very useful way of automating tasks that would be otherwise quite tedious.  Scripts are you friend. In the PROJECT1 directory, there is a file called run_integrate2.sh: #!/bin/sh # run_integrate2.sh # shell script useful for running the program integrate for several cases, # i.e., different orders of Laguerre polynomials used # assumes the existence of file called wieghts.dat_#, where # is the order # of the Laguerre polynomial for n in 2 4 8 16 20 24 28 32 do    f=weights.dat_$n    (./integrate | tail -n 1) << !       $f       $n ! done The first line begins with #!, called a "shebang" followed by /bin/sh, the path to the "Bourne shell", a command interpreter.  It is the program that reads in the commands in your script and decides what to do. There are various interpreters available to you, such as zsh (default on mac os) and bash (Bourne again shell), which are based on sh but with expended functionality.  On some systems, /bin/sh actually is link that points to /bin/bash.  You can check your system by entering ls -l /bin/sh Lines beginning with just a # are comments. You will recognize a loop structure, i.e. a for loop. n is a variable $n retrieves whatever is stored in the variable n The inner part of the loop is a little cryptic.  We will explain everything in the script, but first indicate which of the following statements are true. Q10 The output of a command, usually printed to the screen as "standard output" can be "redirected" into a file. For example, in the project directory, there is a file called README.txt.  You can find all the lines of text that contain the string "dvi" by entering, grep dvi README.txt  which will dump to the screen all (6) lines that contain "dvi". Instead of dumping that information to the screen, you can redirect it to a file called "search_res.txt" with grep dvi README.txt > search_res.txt You get a count of how many lines there are in search_res.txt by entering wc search_res.txt (wc -- word count) which reports the number of lines, words and characters in the file. Rather than using the two-step process of storing the grep results in a file, and then using wc on the file, you can use a "pipe" to feed the output of one command directly as the input to another.  Enter grep dvi README.txt | wc The "|" character is found usually about the ENTER key on a keyboard (but you proabably need to use SHIFT).  With this pipe, we got a count of the number of lines containing "dvi" in README.txt without needing to store a file. Using a single ">" to redirect output will create a new file (if the specified file does not already exist) or overwrite an existing one to store output. Entering grep dvi README.txt > search_res.txt several times in a row will always result in search_res.txt having 6 lines. Using two ">" together, i.e. ">>", will create a new file or overwrite an existing one to store output. Try entering grep dvi README.txt >> search_res.txt grep dvi README.txt >> search_res.txt Each time the command is executed, search_res.txt grows in size. Question: Which of the following would provide the number of files in directory that contain the string "weights" (check all that apply)?  [If you are not sure, just try them on the command line in the PROJECT1 directory.] Q11 Just as standard output of a program can be directed to a file, standard input can be received from a file. For example, run the program "integrate" via ./integrate  It will ask for input, from "standard input".  It asks for the name of a file.  Give it the name "weights.dat_4". It then asks for the "Number of zeros".  Enter 4. If all goes well, the program reads in the right amount of data from the given file and puts out a row of numbers. Instead of typing the input by hand every time, create/edit a file called "input.dat".  The first line should be "weights.dat_4" and the second line should be "4", exactly the two inputs that you fed the program by hand in the previous step. To feed in this information to the program, enter ./integrate < input.dat The program should run as before, except it won't wait for you to provide input because it gets it from input.dat. You can feed in standard input from a file and at the same time redirect output to a different file.  Try ./integrate < input.dat > output.dat The file output.dat will contain the three lines of output that the the program normally writes to the screen. Guess what "tail -n 1 output.dat" does (or just try it). When the above works for you, answer True. Q12 For redirecting output, we had the ">" (overwrite) and ">>" (append).  What would "<<" do?  Well, the answer provides a way of creating a file that provides standard input on the fly, in something called a "Here document". Instead of using ./integrate < input.dat We can instead avoid even creating input.dat in the first place with ./integrate < DVI -> PDF (instead of TeX -> PDF), or install epstopdf. The underlying thing is that latex uses .eps figures, pdflatex uses .pdf figures. To install epstopdf, enter epstopdf The system will not find it, but will tell you what package you need to install. Beaming graphics across from, say, alfven, doesn't work sometimes. On your machine, try export DISPLAY=127.0.0.1:0