At the request of President Yamada of the Nara Institute of Science and Technology, I visited the School of Information Science for two days to evaluate its research activities and provide advice about these activities. My assessment in this report is based on several sources of information:
Although this information is necessarily incomplete, I am confident that it provides a strong basis for an assessment of the School.
This report is divided into several separate pieces:
A) An assessment of the ongoing research activities in the School.
B) A discussion of the evaluation activity itself.
C) Some recommendations for actions that the School and the University could consider.
In advance, let me state very clearly that my assessment and my recommendations are necessarily through the eyes of an outsider of the University, the School, and Japan itself. The criteria for assessment and the recommendations are made with the explicit understanding that I am a foreigner and that I do not understand the constraints imposed by the Japanese educational system, by the Japanese government, and by the Japanese culture.
The research activities of the School of Information Science are rich and are deep. The scope of activities is very large, ranging from highly theoretical topics such as the decidability of finding holes in security protocols to important practical problems such as robotics and virtual reality. The breadth of activities is essential for a School of this size. The depth, of course, is essential for any research program to be successful.
It is not necessary to report on specific research activities in the School. The reports for each laboratory in the self-evaluation are quite useful and are accurate descriptions of the activities. For many laboratories, the English language web pages also provide useful information.
My assessment of these research activities, in addition to confirming the breadth and quality of the research, includes several specific points.
1. There is a significant effort made through the School to participate in international research activities. This is admirable, important, and difficult. There is a variation across the laboratories with respect to their international involvement. Reminding all laboratories of the benefits and importance of international activities would be valuable. In addition, encouraging the laboratories to keep their English language web pages up to date would help in allowing the international world to better understand the excellent research activities within the School.
Separately, I found the English language ability of every person I met to be very strong, stronger than I usually find in Japan. This is important, as it lays the foundation for international activities.
2. There is a noticeable variation in the overall activity levels of the laboratories in the School. Of course, variations are normal and occur for many reasons: changing of staff within a laboratory, shifting of research interests from one topic to another, problems with research equipment, etc. Encouraging each laboratory to continue to produce high-quality research on a regular basis may be helpful in reducing some of this variation.
3. During this visit, as well as previous visits to many universities within Japan, I noticed that there is an almost total absence of research cooperation between the laboratories. Since many difficult research problems require many talents for their solution, this lack of cooperation may limit the kind of research that the School may be able to produce. As one example of this kind of interaction in information science, consider the international conference called ASPLOS, which focuses on the interactions of Architecture, Programming Languages, and Operating Systems. Progress in this area would be very difficult in the current structure of the School (as well as almost every other department and school in information science in Japan).
Finding a mechanism to provide incentives for cooperation among laboratories might be valuable, allowing the School to attack wonderful research problems and find solutions that other academic research groups in Japan cannot achieve.
4. As a minor point, the information about the activities in the laboratories is very effective in describing what the activities are. However, in some cases, they are not as effective in describing why these activities are important. It is important to convey this to other researchers as well as to the laboratory itself, to allow a stronger and consistent attack on the research problems that are of interest.
In addition, it was exciting to see a strong Awards program set up in the School. Universities cannot usually provide strong financial incentives to faculty and students. Therefore, having Awards is an excellent idea, because it is a very inexpensive way to reward high-quality research, teaching and service. In addition, because information science is still a relatively young field, awards help the researchers in the field become more accepted in other companies and universities.
The evaluation process was well-planned and well-designed. The materials that were provided were excellent; the self-study, in particular the summary of the self-evaluation, was especially well-prepared.
My major comment about the evaluation process is that it should be an ongoing activity. Many departments and colleges in the U.S. have yearly external evaluations; in my university, each program is also reviewed in great depth every 10 years. In addition, it is important to have an evaluation team of at least three people: one reason is that each person will have different ideas, and this will make the evaluation more effective; and another reason is that the breadth of research activities is so great in the School that it is important to have evaluators with more collective research knowledge than can be provided by a single person.
In this section I provide some recommendations. Again, it is important to remember that I do not understand the constraints imposed by the Japanese university structure, so many or all of my suggestions may not be applicable in practice. In many of the recommendations I will discuss some mechanisms used in my university and department; but surely many other approaches are as good or better.
1. It is important to find a mechanism to allow the School to move aggressively into emerging research areas. This is very important in information science, since the underlying technology changes so rapidly. Although many theoretical problems remain stable over long periods, the incredible increases in power of processors, memory, and networks, along with fascinating new computing devices, make many of the more practical research topics change quickly. To continue to attract and to excite high-quality students, and to meet the ever-changing needs of industry and society, it is crucial to address emerging areas on a regular basis.
Within my department, we find this difficult to do sometimes. It is often the case that the records of more traditional researchers look stronger when evaluated using traditional means. So we have to work very hard to use subjective measures to assess possible faculty members in emerging areas. We also have to accept more risk in hiring such faculty members.
About five years ago our University hired a new President. (He was hired from another university, which is not unusual in the United States.) One of his major concerns was how to encourage the University to move into new areas and to try riskier projects in research, education, and administration. He decided to tax every budget in the university that is provided by the government 1%. He then requested proposals for the University Initiative Fund (UIF), with a committee set up to make recommendations about which proposals to accept. The UIF is highly competitive: this year there were about 30 proposals, about 10 have passed to the final round, and perhaps five will be funded.
Our Dean also taxes our budgets to provide some limited resources for added flexibility.
2. In information technology, it is important for many researchers to interact with industry. This is happening in many ways at the School, which is excellent. But it is important to keep looking for ways to strength this relationship. When there is a strong relationship, the researchers benefit by developing a deep understanding of the practical problems faced by industry and by resources that industry can provide (including people, data, equipment, and money, while the companies benefit from new ideas and technologies as well as better access to students from the School.
3. The School is in a difficult position: it is necessary to enroll high-quality students, but most strong students in Japan prefer to go to very famous universities. In many ways, the University and the School have already worked very hard to attract strong students. For instance, they provide excellent facilities and computing, they provide more flexibility in courses than is common in Japan, etc. Perhaps there are three ways to attract more strong students. One is to increase the visibility of the University and the School. Two is to continue to make the research in the School of the highest-quality and exciting to potential students. Three is to provide additional flexibility to students; as one small but difficult example, perhaps it would be helpful to allow students to change from one laboratory to another during their studies. (This might also help in building interactions among the laboratories.) Of course, students are often hesitant to move, since it usually adds time to their studies, but perhaps making the flexibility available is still useful.
4. Perhaps there are other kinds of educational programs that the School can create. One example that we recently started was a coursework-only, part-time Professional Masters Program. Students with industrial experience in the information science area take about one course each quarter, at night and some by distance learning, earning their Masters degree in about 2-1/2 or 3 years. Students in the program have as few as 2 or 3 years of experience and as many as 25 years. The university provided new faculty and staff positions to create this program.
Many universities in the United States also have Extension programs, which teach courses (for money) to people who are not enrolled in the university. In some cases, there are single courses (for instance, "How to use the WWW") but in many cases there are collections of courses that allow a person to earn a Certificate from the university. For instance, we have Certificate Programs in Software Product Management, Data Management, Networking, and many more. These courses are entirely self-supporting and may provide extra resources to the School or University.
There are many other issues of importance for which I do not have specific recommendations. Increased self-governance is one example, which we are also facing in the United States: how do we remain accountable to the society and the government while still maintaining the flexibility and control that we need to make progress? Funding issues are another example. In our University, about 20% of the funding comes from the state government, while the other 80% comes from grants and contracts with the federal government and private sources; it may be important for the School to move more in this direction, but that may be difficult given the existing style of funding.
My final note is that it seems that the School of Information Science at NAIST is facing many of the same problems faced by information and computer science schools and departments in the United States: getting strong students, hiring high-quality and versatile faculty, working with industry on important research problems, and understanding an appropriate role in society and with government. I am confident that the School can continue to address these important and difficult problems effectively and intelligently, leading the School into the next millenium.