Introducing Julia/Modules and packages

From Wikibooks, open books for an open world
Jump to: navigation, search
« Introducing Julia
Modules and packages
»
Metaprogramming DataFrames

Modules and packages[edit]

Julia code is organized into files, modules, and packages. Files containing Julia code use the .jl file extension.

Modules[edit]

Related functions and other definitions can be grouped together and stored in modules. The structure of a module is like this:

module MyModule

end

and in between these lines you can put functions, type definitions, constants, and so on.

One or more modules can be stored in a package, and these are managed using the git version control system. Most Julia packages, including the official ones, are stored on GitHub, where each Julia package is, by convention, named with a ".jl" suffix.

Installing modules[edit]

To use an official (registered) Julia module on your own machine, you download and install the package containing the module from the main GitHub site. There's a big list of official packages available at pkg.julialang.org. To download and install one, you give its name to the Pkg.add() function:

julia> Pkg.add("Calculus")
INFO: Installing Calculus v0.2.2
INFO: Package database updated

julia>

(If you are not directly connected to the internet, you need to give the name of a proxy before calling the package installer.)

To find out where packages are installed, type

julia> Pkg.Dir.path()
"/my_home_directory/.julia/v0.6"

Using modules[edit]

After installation, to start using functions and definitions from the module, you tell Julia to make the code available to your current workspace, with the using statement, which accepts the names of one or more installed modules:

julia> using Calculus
julia>

Now, all the definitions in the Calculus module are available for use. If the definitions inside Calculus were exported by the module's author, you can use them without the module name as prefix (because we used using):

julia> derivative(sin, pi/2)
0.0

If the package author(s) don't export the definitions, or if we use import rather than using (See below), you can still access them, but you have to type the module name as a prefix:

julia> Calculus.derivative(sin, pi/2)
0.0

but that's unnecessary in this example, as we've seen.

When you write your own modules, the functions that you choose to export can be used without the module name as prefix, Those that you don't export can still be used, but only if they are prefixed with the module name.

For example, in the module called MyCoolModule, the mycoolfunction() was exported. So the prefix is optional:

julia>using MyCoolModule
julia> MyCoolModule.mycoolfunction()
"this is my cool function"
julia> mycoolfunction()
"this is my cool function"

Inside the module, this function was exported, using the export statement:

module MyCoolModule
export mycoolfunction

function mycoolfunction()
   println("this is my cool function")
end

end

using and import[edit]

import is similar to using, but differs in a few ways, for example, in how you access the functions inside the module. Here's a module with two functions, one of which is exported:

module MyModule
export mycoolfunction

function mycoolfunction()
   println("this is my cool function")
end

function mysecretfunction()
   println("this is my secret function")
end

end

Import the module:

julia> import MyModule
julia> mycoolfunction()
ERROR: mycoolfunction not defined

julia> MyModule.mycoolfunction()
"this is my cool function"

Notice that mycoolfunction() could be accessed only with the module prefix. This is because the MyModule module was loaded using import rather than using. Similarly for mysecretfunction():

julia> mysecretfunction()
ERROR: mysecretfunction not defined

julia> MyModule.mysecretfunction()
this is my secret function

Unlike using, import doesn't let you refer to a number of modules on the same line:

using Color, Calculus, Cairo

Another important difference is when you want to modify or extend a function from another module. You can't use using, you have to import the specific function.

Include, require, reload[edit]

If you want to use code from other files that aren't contained in modules, there are the following functions:

include(pathname) evaluates the contents of the file in the context of the current module, searching relative to the path of the source file from which it is called. This is useful for building code from a number of smaller files.

require(filename) loads the file in the context of the Main module. It first looks in the current working directory, then looks for package code under the current package directory (as reported by Pkg.dir()), then tries paths stored in the global array LOAD_PATH.

reload(filename) is like require, but forces the file to be loaded again. You would probably use this while developing your code.

How does Julia find a module?[edit]

Julia looks for module files in directories defined in the LOAD_PATH variable.

julia> LOAD_PATH
2-element Array{String,1}:
 "/Applications/Julia-0.6.app/Contents/Resources/julia/local/share/julia/site/v0.6"
 "/Applications/Julia-0.6.app/Contents/Resources/julia/share/julia/site/v0.6"

To make it look in other places, add some more using push!:

julia> push!(LOAD_PATH, "/Users/me/julia")
3-element Array{String),1}:
 "/Applications/Julia-0.6.0.app/Contents/Resources/julia/local/share/julia/site/v0.6"
 "/Applications/Julia-0.6.0.app/Contents/Resources/julia/share/julia/site/v0.6"
 "/Users/me/myjuliaprojects"

And, since you don't want to do this every single time you run Julia, put this line into the file ~/.juliarc.jl, which runs each time you start an interactive Julia session.

Julia looks for files in those directories in the form of a package with the structure:

ModuleName/src/file.jl

Or if not in Package form (see below), it will look for a filename that matches the name of your module:

using MyModule

and this would look in the LOAD_PATH for a file called MyModule.jl and load the module contained in that file.

Packages[edit]

The built in module Pkg provides a number of functions for managing the packages you've installed. We've seen Pkg.add():

julia> Pkg.add(packagename)
julia> using Calculus
julia> derivative(cos, 1.0)
-0.8414709847974693

Current status of packages[edit]

The following functions in the Pkg package are very useful, particularly at present when packages are constantly being updated to keep pace with Julia's rapid development.

Pkg.installed() returns the version numbers of your installed packages, in the form of a Julia dictionary (so you could examine the returned dictionary with Julia code):

julia> Pkg.installed()
Dict{String,VersionNumber} with 129 entries:
  "Libz"                 => v"0.2.4"
  "COFF"                 => v"0.0.2"
  "PaddedViews"          => v"0.1.0"
  "AxisAlgorithms"       => v"0.1.6"
  "Juno"                 => v"0.2.7"
  "OffsetArrays"         => v"0.3.0"
  "LegacyStrings"        => v"0.2.1"
  "Lazy"                 => v"0.11.6"
  "IndirectArrays"       => v"0.1.1"
  "ImageFiltering"       => v"0.1.4"
  "TextWrap"             => v"0.2.0"
   ...

Pkg.status() shows additional information about your installed packages.

julia> Pkg.status()
22 required packages:
 - AstroLib                      0.1.0
 - Atom                          0.5.10
 - BenchmarkTools                0.0.8
 - Cairo                         0.3.0
 - Calculus                      0.2.2
  ...
 - Plots                         0.11.2
 - Primes                        0.1.3
 - PyPlot                        2.3.2
 - QuartzImageIO                 0.3.0
 - Requires                      0.4.2
 - Tokenize                      0.1.8              master
 - UnicodePlots                  0.2.3
107 additional packages:
 - ASTInterpreter                0.0.4
 - AbstractTrees                 0.0.4
 ...
 - JuliaParser                   0.7.4
 - Juno                          0.2.7
 - LNR                           0.0.2
 - LaTeXStrings                  0.2.1
 - Lazy                          0.11.6
 - LegacyStrings                 0.2.1
 - Libz                          0.2.4
 - Luxor                         0.8.4+             master
 - MachO                         0.0.4
 - MacroTools                    0.3.6
 - MappedArrays                  0.0.7
 - MbedTLS                       0.4.5
 - Measures                      0.1.0
 - Media                         0.2.7
 - Mellan                        0.0.0-             master (unregistered)
 - Mustache                      0.1.4
 - Mux                           0.2.3
 - NaNMath                       0.2.4
 - NearestNeighbors              0.2.0
 - ObjFileBase                   0.0.4
 - OffsetArrays                  0.3.0
 - OhMyREPL                      0.1.0+             master
 - PaddedViews                   0.1.0
 - PhilipsHue                    0.0.0-             master (unregistered)
 - PlotThemes                    0.1.3
 - PlotUtils                     0.4.1
  ...
 - TexExtensions                 0.0.3
 - TextWrap                      0.2.0
 - Thebes                        0.0.0-             master (unregistered, dirty)
 - TiledIteration                0.0.2
 - URIParser                     0.1.8
 - VT100                         0.1.0
 - WebSockets                    0.2.1
 - WoodburyMatrices              0.2.2
 - ZMQ                           0.4.2

You can see that some of the packages are not in their normal, default state: those labelled "master" have been specifically checked out from github, rather than simply downloaded. The "unregistered" packages are not official Julia packages (they haven't been added to [METADATA]. And the "dirty" packages currently have changes that haven't even been committed to their locally stored master.

To update packages that are out of date compared with the official master versions stored on GitHub, use Pkg.update(). You can update everything:

julia> Pkg.update()
INFO: Updating METADATA...
INFO: Upgrading DataStreams: v0.0.4 => v0.0.5
INFO: Upgrading NullableArrays: v0.0.2 => v0.0.3
INFO: Upgrading SQLite: v0.3.1 => v0.3.2
julia>

Or you can just update individual packages. All other packages that they depend on are also updated automatically.

julia> Pkg.update("Documenter")
INFO: Updating METADATA...
INFO: Updating cache of FileIO...
INFO: Updating cache of Compat...
INFO: Updating cache of JSON...
INFO: Updating cache of DocStringExtensions...
INFO: Computing changes...
INFO: Upgrading Compat: v0.8.7 => v0.8.8
INFO: Upgrading DocStringExtensions: v0.1.0 => v0.2.0

Structure of a package[edit]

Julia uses git for organizing and controlling packages. By convention, all packages are stored in git repositories, with a ".jl" suffix. So the Calculus package is stored in a Git repository called Calculus.jl. This is how the Calculus package is organized in terms of files on disk:

Calculus.jl/                                       # this is the main package directory for the Calculus package
  src/                                             # this is the subdirectory containing the source
    Calculus.jl                                    # this is the main file - notice the capital letter
      module Calculus                              # inside this file, declare the module name
        import Base.ctranspose                     # and import other packages
        export derivative, check_gradient,         # export some of the functions defined in this package
        ...
        include("derivative.jl")                   # include the contents of other files in the module
        include("check_derivative.jl")             
        include("integrate.jl")
      end                                          # end of Calculus.jl file
    derivative.jl                                  # this file contains code for working with derivatives, 
      function derivative()                        #      and is included by Calculus.jl
        ...
      end
        ...
    check_derivative.jl                            # this file concentrates on derivatives, 
      function check_derivative(f::...)            #      and is included by "include("check_derivative.jl")" in Calculus.jl
        ...
      end
      ...
    integrate.jl                                   # this file concentrates on integration, 
      function adaptive_simpsons_inner(f::Function......   # and is included by Calculus.jl
        ...
      end
      ...
    symbolic.jl                                    # concentrates on symbolic matters; included by Calculus.jl
      export processExpr, BasicVariable, ...       # these functions are available to users of the module
      import Base.show, ...                        # some Base functions are imported, 
      type BasicVariable <: AbstractVariable              #            so that more methods can be added to them
        ...
      end
      function process(x::Expr)
        ...
      end
      ...     
  test/                                            # this directory contains the tests for the Calculus module
    runtests.jl                                    # this file runs the tests
      using Calculus                               # obviously the tests use the Calculus module... 
      using Base.test                              # and the Base.test module... 
      tests = ["finite_differerence", ...          # the test file names are stored as strings... 
      for t in tests
        include("$(t).jl")                         # ... so that they can be evaluated in a loop 
      end
      ...
    finite_difference.jl                           # this file contains tests for finite differences, 
      @test ...                     #        its name is included and run by runtests.jl
      ...

Global variables and scope[edit]

It's common to want to control access to variables. Sometimes you want to define a variable so that every function in the module can use the same value. To do this, you can use the global keyword when defining the variable.

Here's an example of a module called Sundial, and how you can access the variable called latitude, which presumably will be useful for a number of different functions inside the module.

module Sundial
global latitude = 52
export latitude, get_lat, set_lat, use_lat
function get_lat()
   println("in Sundial.get_lat, latitude is $latitude")
end

function set_lat(lat)
   global latitude
   latitude = lat
   println("in Sundial.set_lat, latitude is now $latitude")
end

function use_lat_fail()
   try
     println("in Sundial.use_lat_fail, latitude is $latitude")
   catch
     println("couldn't find it")
   end
   
   latitude = 0
   println("in Sundial.use_lat_fail, latitude was set to 0, is $latitude")
end

function use_lat()
   global latitude
   try
     println("in Sundial.use_lat, latitude is $latitude")
   catch
     println("couldn't find it")
   end
   
   latitude = 0
   println("in Sundial.use_lat, latitude was set to 0, is $latitude")
end
end

The second line of the module uses the global keyword and sets the value of the variable to 52. Other functions in the module can access this variable if they use the global keyword. These functions, and the name of the variable, are exported using this line:

export latitude, get_lat, set_lat, use_lat

Before you load the module into Julia, there is no variable called latitude:

julia> latitude
ERROR: latitude not defined

After adding the module:

julia> using Sundial

the value of latitude is available, and can be used without the prefix, because the module exported it and because we loaded the module with using:

julia> latitude
52

It's also available with its module prefix:

julia> Sundial.latitude
52

but we can't change its value, because we're not "in" the Sundial module, as the error message tells us:

julia> Sundial.latitude = 2
ERROR: cannot assign variables in other modules

Our get_lat() function can access the variable's value:

julia> Sundial.get_lat()
in Sundial.get_lat, latitude is 52

Although you can read the value easily, you can't set it. For example:

julia> latitude = 40
Warning: imported binding for latitude overwritten in module Main
40

The warning here tells you that there are now two variables called latitude, a new one you've created in the Main context, and the existing one in the Sundial context. They have different values:

julia> latitude
40

julia> Sundial.latitude
52

julia> Main.latitude
40

The get_lat() and set_lat() functions provided by the module allow you to access and change the values from anywhere:

julia> Sundial.get_lat()
in Sundial.get_lat, latitude is 52

julia> Sundial.set_lat(45)
in Sundial.set_lat, latitude is now 45

julia> Sundial.get_lat()
in Sundial.get_lat, latitude is 45

The set_lat() function used the global keyword at the start of the definition. This means that code inside the function definition can change the value.

Without the global keyword, as in the use_lat_fail() function, any attempt to change the module's latitude variable fails.

julia> Sundial.use_lat_fail()
couldn't find it
in Sundial.use_lat_fail, latitude was set to 0, is 0

But wait: the use_lat_fail() function appears to work - it says that latitude was set to 0. This is a new, local, latitude symbol created just for the duration of the user_lat_fail() function. However, you can quickly check to reassure yourself that the "real" module's variable has not been set to 0:

julia> Sundial.get_lat()
in Sundial.get_lat, latitude is 45

To change the variable, the use_lat() (without the '_fail') function can set the module's latitude variable to 0, thanks to the use of the global keyword:

julia> Sundial.use_lat()
in Sundial.use_lat, latitude is 45
in Sundial.use_lat, latitude was set to 0, is 0

The warning you received above, about overriding the imported binding in Main, prepares you for the following:

julia> latitude
40

julia> Sundial.latitude
0

julia> Main.latitude
40

julia> Sundial.get_lat()
in Sundial.get_lat, latitude is 0

The Main context's version of latitude, also available without the Main prefix, is still 40 because the use_lat() function changed the module's latitude variable to 0. You can't now access the module's version of latitude without using the Sundial prefix.

The correct way to set the value of the module's global variable is to use set_lat():

julia> Sundial.set_lat(45)
in Sundial.set_lat, latitude is now 45

julia> latitude
60

julia> Sundial.latitude
45

julia> Main.latitude
60

julia> Sundial.get_lat()
in Sundial.get_lat, latitude is 45

The 'wrong' one is still 60, but the module's version is correctly set to 45.