Japanese History/The Jomon Period
An Overview of the Jomon Period 
I am not aware of any evidence for very early human settlement in Japan. There were definitely modern humans, Homo Sapiens, in Japan during the latter parts of the Pleistocene period, the great ice ages. Japan was connected to the mainland at several periods during that era, both to the southern end of Korea and to Siberia, and the animal and plant and human populations seem to have been identical to those in the adjacent parts of the mainland. The sea levels were very much lower than in our age (hundreds of feet lower) and it is likely that most human remains in this area are under deep water.
When the islands were disconnected from Asia for the final time things inevitably changed. The megafauna (elephants, tigers, and giant deer) died out, the plant life began to change radically, and the sea levels rose rapidly. The people trapped on the islands had to change their way of life. Similar things happened almost everywhere in the temperate zones of the world. In Europe the change is often identified as a transition from paleolithic to mesolithic, from a life hunting large animals that roamed in large herds on open lands, to a life hunting smaller animals that lived alone or in small groups hidden in thick forests. In Japan, this is the transition to Jomon. The name "mesolithic" is not used in Japan, because in Europe it refers to particular types of stone tools that do not occur in Japan. The name Jomon is a translation to Japanese of the English term "cord marked" which was applied to describe the general appearance of Jomon pottery by its original (American) discoverer.
Jomon Culture 
The Jomon culture is old. The oldest radiocarbon dates are presently about 10,000 BC and older, undated, sites are known. When this was realized in the 1950's there was considerable excitement, because even the earliest Jomon sites show the use of pottery. It was then and still remains today after many discoveries of early pottery elsewhere in the world the oldest pottery known. The initial discoveries were of a relatively late phase (2500 BC or so) when the Jomon people were living in villages and making art pottery as well as useful items, and because of the pottery and the nature of the stone tools that were being made, it was first announced that the Jomon culture was "neolithic". However, in the world at large neolithic usually is taken to refer to agricultural societies, and the Jomon people were not farmers. They were among a quite small number of pre-agricultural peoples who lived in an environment that was rich enough that people could settle in permanent villages without farming. As a result, Japanese archaeologists prefer to avoid the use of the term neolithic and simply use Jomon.
Jomon culture lasted for a long time, isolated on the Japanese islands. Books in English will tell you that the transition to the subsequent Yayoi culture occurred sometime around 400 or 300 BC. The actual date is more like 1000 or 900 BC (a very recent discovery discussed later). From the Jomon perspective this is an insignificant change. This was a culture that lasted for about 10,000 years. During that time the ecology of the Japanese islands was constantly changing and the Jomon people had to change with it. At any particular time there was only a relatively small portion of the country that could support a population dense enough for villages, and in different periods this portion was not always in the same place. This extremely long life tells us that conditions in Japan were not good for the independent development of farming. The forests were heavy and there were no native edible plants with revolutionary potential. The most important single plant food source throughout Jomon was nuts gathered in the forests.
The richest phase of all was "Middle Jomon" between roughly 3000 BC and 2000 BC. The total population has been estimated (based on counting house sites) at from 50,000 to 100,000 people, the overwhelming majority of whom were concentrated in eastern Japan in modern Tokyo and regions extending westward into nearby hills where hunting appears to have been particularly successful, mainly for Japanese "shika" deer and wild pigs. Shell mounds are common, and fishing, including deep sea fishing, was important.
Late Jomon Period 
After 2000 BC the climate changed considerably (getting colder) which affected the makeup of the trees of the forest, ending the Middle Jomon. The population crashed and moved to new areas in the Late Jomon. Many different kinds of nuts were relied upon at different times as the climate varied. Some types of nuts are "ready to eat" and others require processing to remove tannins, and this greatly affected the density of the population at different times. The east now lost easy to handle acorns and had to shift to difficult to handle horse chestnuts. A coastal area northeast of Tokyo received a culture which clearly continued the traditions of middle Jomon. It is this culture that provides the most spectactular Jomon pottery, the "Kamegaoka style", and other art, justifiably popular in museums. A significantly different culture emerged in Kyushu with a quite different kind of pottery, a plain, black ware. This culture has attracted a great deal of attention because many archaeologists see signs of incipient agriculture, though it definitely continued to rely on gathered wild nuts as the primary source of plant food. The total population was quite small. I have seen estimates of 5,000 or 6,000 people for Kyushu, with maybe twice as many in the northeast. Large parts of Kyushu are covered by volcanic rocks and cannot support the density of forests that existed elsewhere and this restricted the population.
From Jomon to Yayoi Culture 
The transition from Jomon to Yayoi occurred in northern Kyushu. The Yayoi culture (named after a neighborhood in Tokyo where the first remains were recognized) was made up of rice farmers using an already mature technology involving irrigation. By the end of the period they were making bronze objects of sufficient quality that it is possible to argue about whether certain ones were made in Japan or China, and also iron tools and weapons. The biggest topic of discussion is whether the Jomon people acquired these new technologies (from Korea) and converted themselves into Yayoi people, or whether the Yayoi people were immigrants. I will explore the arguments, but it is pretty certain that the Yayoi people were immigrants. The Jomon population of Kyushu was very small and the early Yayoi population was comparatively large. There is evidence that Yayoi practices spread rapidly eastward as far as the area of modern Nagoya, where there was a pause. Its spread from there to the northeast appears to have been accomplished in large part by converting Jomon people to the new way of life. The pottery of the early Yayoi culture in the northeast shows characteristic Jomon decorative elements that are completely unknown in the west.
Probably the most important fact in this context is that there was a widespread culture in southern Korea, known as "Mumun", after its characteristic pottery, that was extremely similar to Yayoi, so much so that in the early period it is sometimes very difficult to tell them apart. This has become much better known in recent years as South Korea is now rich enough to spend significant money on archaeology, and it is pretty clear that the arrival of Yayoi in Japan was the result of immigration of people from Mumun Korea. It was formerly thought that the two were about the same age so that both might have arrived together from somewhere on the Chinese coast (though no one could suggest where), but it is now clear that Mumun is significantly older than Yayoi.
Further Reading 
I am an historian not an archaeologist, and I do not intend to attempt a real description of Jomon culture or trace its evolution through multiple phases. For a good, not excessively long, treatment in English I recommend:
Imamura Keiji 1996, Prehistoric Japan: new perspectives on insular East Asia (Honolulu: University of Hawai'i Press)
This book also covers Yayoi and the beginnings of the following Kofun period. It was written before the recent revolution in dating I have referred to above, for which you must look in the chapter on the Yayoi period.