Clipper Tutorial: a Guide to Open Source Clipper(s)/Working With Databases

From Wikibooks, open books for an open world
Jump to navigation Jump to search

Working with Databases[edit]

Let's return to the Wikipedia entry Database application.

Different kinds of database applications exist as well. If you did store your friend's phone numbers and addresses into a word processor, you would have what someone calls a Free-Form Database (however, a similar expression is an oxymoron in computer science) - myBase®, askSam®, Personal Knowbase®, MyInfo®, Info Select®, and GeneralKB® are a bunch of specialized free-form database application, which actually means PIM (Personal information manager). Now, a word processor lets us search the information, but other operations, such as sorting them in alphabetical or numerical order, cannot be done automatically by a word processor.

What about attempting to store it into a spreadsheet? We may use one column for the name, one for the surname, one for the telephone number, one for the city. This quick database, stored in a spreadsheet, may be searched and sorted: for example, we can sort it by city and person's name in alphabetical order. This is a flat database, http://www2.research.att.com/~gsf/man/man1/cql.html: a flat database is a sequence of newline terminated records of delimiter separated fields, and a spreadsheet shows its limits in data entry and reporting (if you did want to use the data in your table to print out addresses on envelopes a spreadsheet is not a good tool). An example is MyDatabase (http://www.pcmag.com/article2/0,2817,760833,00.asp).

Spreadsheets are much better to do accounting: how much harder a book-keeper's work would be if his data were stored in a wordprocessing program? The purpose here is to have our data structured in a certain way: all the costs in a place, all earnings in another.

Before 1970 complex databases were managed using Hierarchical Databases (very little information is needed about them - see for example http://www.extropia.com/tutorials/sql/hierarchical_databases.html and http://people.cs.pitt.edu/~chang/156/14hier.html). An example of a hierarchical database is IBM IMS (Information Management System, which was developed during the mid-1960s for applications in the aerospace industry). Their implementation is based on trees, a hierarchical data structure. Hierarchical Databases and Network Databases together form what today are referred to as Legacy Database Systems. Network databases were born as an extension to the programming language COBOL by the Conference on Data Systems Languages (CODASYL). The hierarchical data model is based on data structures called graphs.

Today's standard is the Relational Database (RDBMS), which is "a database with relationships between more than one table of records based on common fields". We will speak of them in some detail, but we will briefly mention the fourth approach: Object Oriented Databases. These databases store objects (in the same sense the word is used in the expression object-oriented programming). They're not much used, mostly because objects are more complex than the simple fields a relational database stores in its tables. More information on the topic at http://www.odbms.org/odmg-standard/.

The Wikipedia entry about DBase reads: «dBase is application development language and integrated navigational database management system which Ashton-Tate labeled as "relational" but it did not meet the criteria defined by Dr. Edgar F. Codd's relational model». Codd's criteria (the so-called 12 rules, which really are 13 because the rule numbered '0' actually exists) are so strict that in practice a true relational database system does not even exist, but the point is that dBase accessed databases in another way, so that it's considered a Navigational Database (which works in a way that simulates relational databases).

http://www.databasedev.co.uk/design_basics.html

DBF Files in Other Languages[edit]

Because of the great success of dBase and its, the DBF file format became an industry standard. Many other database programs have used them to store data, like Lotus Approach. We also have many little utilities to view and convert to other formats these files. Here is a bunch of URLs: https://dbfview.com/, http://www.alexnolan.net/software/dbf.htm, https://dbfviewer.com/en/, https://www.dbf2002.com/, http://www.whitetown.com/dbf2sql/ («DBF to SQL Converter allows you to convert your dbf files to SQL script. Personal license $29.95», but compare https://www.vlsoftware.net/exportizer/). And is so widely used that interfaces for working with it are available for various languages, for example:

Well, now we will see how to work with DBF files the way it was intended.

Making a first database and recording some data[edit]

A verbose way[edit]

 && it was done this way at the Dot Prompt
 && we can type this interactively in hbrun
 CREATE TMPNAMES
 USE TMPNAMES
 APPEND BLANK
 REPLACE FIELD_NAME WITH "NAME"
 REPLACE FIELD_TYPE WITH "C"
 REPLACE FIELD_LEN WITH 15
 APPEND BLANK
 REPLACE FIELD_NAME WITH "ADDRESS"
 REPLACE FIELD_TYPE WITH "C"
 REPLACE FIELD_LEN WITH 30
 CLOSE
 CREATE NAMES FROM TMPNAMES && https://www.itlnet.net/programming/program/Reference/c53g01c/ngc785e.html
 ERASE TMPNAMES.DBF && we get rid of the temporary file

The code above created a DBF file, names.dbf, to be used by the following code. It will add a record to the DBF file. It is equivalent to the "First Sample Program" of my old PC GUIDE, which missed a line that is necessary in modern xBase.

 CLEAR
 ? "First Sample Program"
 SELECT 1
 USE NAMES
 APPEND BLANK
 REPLACE NAME WITH "MIKE BROWN"
 REPLACE ADDRESS WITH "ROME STREET, 56"
 CLOSE && this line is missing in my PC GUIDE but is needed in a compiled Harbour program
 QUIT

The CLOSE command is equivalent to the dbCloseArea() function, which closes a work area: Pending updates are written, pending locks are released.

A more concise way[edit]

The short code below does the same work of the two pieces of code of the previous section (it only produces a different file name, namesdb.dbf instead of names.dbf).

 local aStruct := { { "NAME",    "C", 15, 0 }, ;
                    { "ADDRESS", "C", 30, 0 }}
 REQUEST DBFCDX
 dbCreate( "namesdb", aStruct, "DBFCDX", .t., "NAMESDB" )
 && http://www.fivetechsoft.com/harbour-docs/api.html
 USE NAMESDB
 NAMESDB->(DbAppend())
 NAMESDB->NAME := "MIKE BROWN"
 NAMESDB->ADDRESS := "ROME STREET, 56"

This example uses the alias operator, ->. http://www.ousob.com/ng/clguide/ngcf412.php

The alias->field_name notation is used to allow access to fields of databases that are loaded but not active. The alias can be specified with the work area number (e.g. 2->std_id), with the work area alias (e.g. B->std_id), or with the database name (e.g. STUDENTS->std_id).

The result of this code is a file named namesdb.dbf. Informations about how DBF files are can be find at DBF File structure, http://www.dbf2002.com/dbf-file-format.html, where we find this list of Field type:

  • C – Character
  • Y – Currency
  • N – Numeric
  • F – Float
  • D – Date
  • T – DateTime
  • B – Double
  • I – Integer
  • L – Logical
  • M – Memo
  • G – General
  • C – Character (binary)
  • M – Memo (binary)
  • P – Picture
  • + – Autoincrement (dBase Level 7)
  • O – Double (dBase Level 7)
  • @ – Timestamp (dBase Level 7)

My PC GUIDE showed how a .dbf file is made with the DataBase Utility DBU. Clones of this utility are FiveDBU (with source code) at https://code.google.com/archive/p/fivewin-contributions/downloads, DBF Viewer Plus at http://www.alexnolan.net/software/dbf.htm, CLUT at http://www.scovetta.com/archives/simtelnet/msdos/clipper. Harbour includes its own HbDBU (the source is in \hb32\addons\hbdbu) and a component IdeDBU of HbIDE (the other two components are IdeEDITOR and IdeREPORTS).

From https://code.google.com/archive/p/fivewin-contributions/downloads we can get fivedbu_20130930.zip (New FiveDBU version with enhancements on ADO fields editing). It supports ADO, 3 RDD (DBFNTX, CBFCDX and RDDADS) and 6 languages - select "Bases de datos -> Preferencias -> Lenguaje: Inglés" to have it in English.

Let us see what is in our little file so far.

 USE NAMES
 LIST DATE(), TIME(), NAME, ADDRESS

Database Design Issue: the First Normal Form (1NF)[edit]

The work done in the previous section was intended to exactly reproduce the database presented in my PC GUIDE. There are, however, drawbacks: having only one NAME field, this database cannot sort its data on the last name. Also, a careless user might insert the data of some people with the last name first, and some other data with the first name last. When designing a database precautions should be taken of these possibilities. The first normal form (http://www.1keydata.com/database-normalization/first-normal-form-1nf.php, http://www.sqa.org.uk/e-learning/SoftDevRDS02CD/page_14.htm) requires you to define fields whose information cannot be divided into smaller parts. So, instead of a NAME field, we should have a FIRST_NAME and LAST_NAME fields. Complying to the first normal form, our little database would be on the on the right track to being a normalized database.

Designing the database is an essential part of the work and it is not always obvious how it should be done. See https://www.ntu.edu.sg/home/ehchua/programming/sql/relational_database_design.html.

A graphical device used to help is database design are the Entity-Relationship Diagrams: https://www.lucidchart.com/pages/er-diagrams, https://www.guru99.com/er-diagram-tutorial-dbms.html.

Complicating our simple database[edit]

Harbour contains a file named test.dbf. Launch hbrun in its directory and type in

use test
browse()

At this point, we see that it is a 500 record table. Move around with the cursor keys and, when you're finished, punch the Esc key to quit this interactive table browser and editor. To get the record number of a person called Ted issue:

locate for first="Ted"
? recno()

Here is the testdbf.prg source from \hb30\tests. It should be discussed in detail. It is a GPL piece of code poorly commented.

 /*
  * $Id: testdbf.prg 1792 1999-11-10 10:17:19Z bcantero $
  */

 function main()

   local nI, aStruct := { { "CHARACTER", "C", 25, 0 }, ;
                          { "NUMERIC",   "N",  8, 0 }, ;
                          { "DOUBLE",    "N",  8, 2 }, ;
                          { "DATE",      "D",  8, 0 }, ;
                          { "LOGICAL",   "L",  1, 0 }, ;
                          { "MEMO1",     "M", 10, 0 }, ;
                          { "MEMO2",     "M", 10, 0 } }

   REQUEST DBFCDX

   dbCreate( "testdbf", aStruct, "DBFCDX", .t., "MYALIAS" )

   ? "[" + MYALIAS->MEMO1 + "]"
   ? "[" + MYALIAS->MEMO2 + "]"
   ? "-"
   MYALIAS->( dbAppend() )
   MYALIAS->MEMO1 := "Hello world!"
   MYALIAS->MEMO2 := "Harbour power"
   ? "[" + MYALIAS->MEMO1 + "]"
   ? "[" + MYALIAS->MEMO2 + "]"
   MYALIAS->( dbAppend() )
   MYALIAS->MEMO1 := "This is a test for field MEMO1."
   MYALIAS->MEMO2 := "This is a test for field MEMO2."
   ? "[" + MYALIAS->MEMO1 + "]"
   ? "[" + MYALIAS->MEMO2 + "]"
   MYALIAS->NUMERIC := 90
   MYALIAS->DOUBLE := 120.138
   ? "[" + Str( MYALIAS->DOUBLE ) + "]"
   ? "[" + Str( MYALIAS->NUMERIC ) + "]"

   ? ""
   ? "Press any key..."
   InKey( 0 )

   ? ""
   ? "Append 50 records with memos..."
   for nI := 1 to 50
      MYALIAS->( dbAppend() )
      MYALIAS->MEMO1 := "This is a very long string. " + ;
                        "This may seem silly however strings like this are still " + ;
                        "used. Not by good programmers though, but I've seen " + ;
                        "stuff like this used for Copyright messages and other " + ;
                        "long text. What is the point to all of this you'd say. " + ;
                        "Well I am coming to the point right now, the constant " + ;
                        "string is limited to 256 characters and this string is " + ;
                        "a lot bigger. Do you get my drift ? If there is somebody " + ;
                        "who has read this line upto the very end: Esto es un " + ;
                        "sombrero grande rid¡culo." + Chr( 13 ) + Chr( 10 ) + ;
                        "/" + Chr( 13 ) + Chr( 10 ) + "[;-)" + Chr( 13 ) + Chr( 10 )+ ;
                        "\"
   next
   MYALIAS->( dbCommit() )

   ? "Records before ZAP:", MYALIAS->( LastRec() )
   ? "Size of files (data and memo):", Directory( "testdbf.dbf" )[1][2], ;
      Directory( "testdbf.fpt" )[1][2]
   MYALIAS->( __dbZap() )
   MYALIAS->( dbCommit() )
   ? "Records after ZAP:", MYALIAS->( LastRec() )
   ? "Size of files (data and memo):", Directory( "testdbf.dbf" )[1][2], ;
      Directory( "testdbf.fpt" )[1][2]
   ? "Value of fields MEMO1, MEMO2, DOUBLE and NUMERIC:"
   ? "[" + MYALIAS->MEMO1 + "]"
   ? "[" + MYALIAS->MEMO2 + "]"
   ? "[" + Str( MYALIAS->DOUBLE ) + "]"
   ? "[" + Str( MYALIAS->NUMERIC ) + "]"
   ? "Press any key..."
   InKey( 0 )
   dbCloseAll()

   dbCreate( "testdbf", aStruct,, .t., "MYALIAS" )

   for nI := 1 to 10
      MYALIAS->( dbAppend() )
      MYALIAS->NUMERIC := nI
      ? "Adding a record", nI
      if nI == 3 .or. nI == 7
         MYALIAS->( dbDelete() )
         ? "Deleting record", nI
      endif
   next
   MYALIAS->( dbCommit() )

   ? ""
   ? "With SET DELETED OFF"
   ? "Press any key..."
   InKey( 0 )

   MYALIAS->( dbGoTop() )
   do while !MYALIAS->( Eof() )
      ? MYALIAS->NUMERIC
      MYALIAS->( dbSkip() )
   enddo

   SET DELETED ON
   ? ""
   ? "With SET DELETED ON"
   ? "Press any key..."
   InKey( 0 )

   MYALIAS->( dbGoTop() )
   do while !MYALIAS->( Eof() )
      ? MYALIAS->NUMERIC
      MYALIAS->( dbSkip() )
   enddo

   ? ""
   ? "With SET DELETED ON"
   ? "and  SET FILTER TO MYALIAS->NUMERIC > 2 .AND. MYALIAS->NUMERIC < 8"
   ? "Press any key..."
   InKey( 0 )

   MYALIAS->( dbSetFilter( { || MYALIAS->NUMERIC > 2 .AND. MYALIAS->NUMERIC < 8 }, ;
                           "MYALIAS->NUMERIC > 2 .AND. MYALIAS->NUMERIC < 8" ) )
   MYALIAS->( dbGoTop() )
   do while !MYALIAS->( Eof() )
      ? MYALIAS->NUMERIC
      MYALIAS->( dbSkip() )
   enddo

   SET DELETED OFF
   ? ""
   ? "With SET DELETED OFF"
   ? "and  SET FILTER TO MYALIAS->NUMERIC > 2 .AND. MYALIAS->NUMERIC < 8"
   ? "Press any key..."
   InKey( 0 )

   MYALIAS->( dbSetFilter( { || MYALIAS->NUMERIC > 2 .AND. MYALIAS->NUMERIC < 8 }, ;
                           "MYALIAS->NUMERIC > 2 .AND. MYALIAS->NUMERIC < 8" ) )
   MYALIAS->( dbGoTop() )
   do while !MYALIAS->( Eof() )
      ? MYALIAS->NUMERIC
      MYALIAS->( dbSkip() )
   enddo

   ? "dbFilter() => " + dbFilter()
   ? ""

   ? "Testing __dbPack()"
   ? "Records before PACK:", MYALIAS->( LastRec() )
   ? "Size of files (data and memo):", Directory( "testdbf.dbf" )[1][2], ;
      Directory( "testdbf.dbt" )[1][2]
   SET FILTER TO
   MYALIAS->( __dbPack() )
   MYALIAS->( dbCommit() )
   ? "Records after PACK:", MYALIAS->( LastRec() )
   ? "Size of files (data and memo):", Directory( "testdbf.dbf" )[1][2], ;
      Directory( "testdbf.dbt" )[1][2]
   ? "Press any key..."
   InKey( 0 )
   ? "Value of fields:"
   MYALIAS->( dbGoTop() )
   do while !MYALIAS->( Eof() )
      ? MYALIAS->NUMERIC
      MYALIAS->( dbSkip() )
   enddo
   ? ""

   ? "Open test.dbf and LOCATE FOR TESTDBF->SALARY > 145000"
   ? "Press any key..."
   InKey( 0 )
   dbUseArea( ,, "test", "TESTDBF" )
   locate for TESTDBF->SALARY > 145000
   do while TESTDBF->( Found() )
      ? TESTDBF->FIRST, TESTDBF->LAST, TESTDBF->SALARY
      continue
   enddo
   ? ""
   ? "LOCATE FOR TESTDBF->MARRIED .AND. TESTDBF->FIRST > 'S'"
   ? "Press any key..."
   InKey( 0 )
   dbUseArea( ,, "test", "TESTDBF" )
   locate for TESTDBF->MARRIED .AND. TESTDBF->FIRST > 'S'
   do while TESTDBF->( Found() )
      ? TESTDBF->FIRST, TESTDBF->LAST, TESTDBF->MARRIED
      continue
   enddo

 return nil

Input Mask[edit]

A simple data base input mask (from the Wikipedia Clipper entry):

USE Customer SHARED NEW
clear
@  1, 0 SAY "CustNum" GET Customer->CustNum PICT "999999" VALID Customer->CustNum > 0
@  3, 0 SAY "Contact" GET Customer->Contact VALID !empty(Customer->Contact)
@  4, 0 SAY "Address" GET Customer->Address
READ

RDDs: What Functions Are Available?[edit]

http://harbourlanguage.blogspot.de/2010/06/understanding-harbour-rdd.html

ADO RDD: Much Ado About Nothing[edit]

https://searchsqlserver.techtarget.com/definition/ActiveX-Data-Objects http://cch4clipper.blogspot.com/2009/10/using-adordd-with-harbourxharbour.html

Case Study: Checkbook Balancing[edit]

Deleting records[edit]

? LASTREC()
DELETE RECORD 4
PACK
? LASTREC()

In this piece of code, the command DELETE marks the fourth record for deletion. But the file is not altered, not even by a CLOSE command. The PACK command actually removes the records marked for deletion (and also makes some additional work). The RECALL command removes the deleted flags. The function DELETED() returns .T. if the current record is marked for deletion, .F. if not.

The PACK command, which does the actual deletion of data from the table, PACK requires that the current database be USEd EXCLUSIVEly. If this condition is not met when the PACK command is invoked, CA-Clipper generates a runtime error. Additional work that PACK does is to update indexes on the table it alters (if any).

The commands DELETE ALL and PACK are executed by a single command called ZAP.

     &&  This example demonstrates a typical ZAP operation in a network
     &&   environment:

        USE Sales EXCLUSIVE NEW
        IF !NETERR()
           SET INDEX TO Sales, Branch, Salesman
           ZAP
           CLOSE Sales
        ELSE
           ? "Zap operation failed"
           BREAK
        ENDIF

An Indexed Example[edit]

USE Clients NEW
INDEX ON Name TO Clients UNIQUE

Suppose a table containing these data:

FSTNAME	LSTNAME
John   	Doe
John   	Doe
John   	Doe
Jane   	Doe

We can create a little index file with this piece of code:

SELECT 1
USE ind
? FILE("ind.ntx")
INDEX ON FstName TO ind
? FILE("ind.ntx") // we verify that a NTX file has been created

Set Relation - Working with more than one table[edit]