Hints for Organizing a Successful Doctoral Symposium

An Experience Report

Claudia Raibulet

and Eila Ovaska

Dipartimento di Informatica, Sistemistica e Comunicazione, University of Milano-Bicocca, Viale Sarca, 336, Milan, Italy

VTT Technical Research Centre, Kaitovayla, 1, Oulu, Finland

Keywords: Doctoral Symposium, Phd Students, Software Engineering.

Abstract: This paper describes our joint experience in organizing a Doctoral Symposium co-located with one of the

main software engineering conferences. It presents the issues we addressed during the organization of the

symposium, as well as during the symposium. This paper is addressed (1) to the organizers of Doctoral

Symposiums aiming to provide a recipe with the main ingredients and preparation steps and their related

significance in the entire organization process, as well as (2) to the PhD students providing our feedback on

the expectations, evaluation, and presentation of their contributions.

1 INTRODUCTION

PhD students in software engineering represent the

pool of researchers who aim to follow an academic

carrier (and, thus, teach students how to do software

engineering) or to join an industrial reality (and,

thus, follow the engineering of large and complex

software) (Almi, 2011); (Subrahmanyam, 2009);

(Villavicencio, 2012); (XueYun, 2010). In this

perspective, PhD students should be able to make a

research plan of high quality and to follow its

implementation from all the points of view

(technical and organizational/managerial).

Moreover, they should be able to properly interact

with all the stakeholders of their research work (e.g.,

in this particular context, experts in the research

domain who are not the requesters/contractors of the

project and do not know any detail or history of the

project, but who may "buy" the idea and/or the

solution).

The Doctoral Symposiums organized in the

context of the main conferences give the PhD

students the possibility to present their research

projects (and research results) and to have a valuable

feedback from experts in the software engineering

domain. A presentation in the context of a Doctoral

Symposium should be seen as a preview of a PhD

dissertation.

This paper presents the experience of the co-

chairs of a Doctoral Symposium co-located in the

context of one of the most important software

engineering conferences: European Conference on

Software Architectures (ECSA, 2013).

1.1 Why This Paper?

The motivation beyond this paper is twofold.

The first concerns the organization of the

Doctoral Symposium. The co-chairs have experience

in organizing such types of events (e.g., conferences,

workshops, doctoral symposiums). Concerning the

Doctoral Symposiums, we have gathered quite

different experiences in organizing different

symposiums, differences coming from the research

area and from the co-located conferences. Hence, in

this paper we aim to summarize the hints and the

lessons learned from the organization of such events

by providing as an example our experience in

organizing this Doctoral Symposium (Ovaska,

2010).

The second concerns our feedback through this

paper to the PhD students. In our opinion it is very

useful for them to know the rules of the game in

order to prepare and present a meaningful research

plan, convince the reviewers and the attendees of the

significance of their work, justify the results and

value of the proposed work, and make a

capturing/interesting presentation in front of the

software domain experts. This will help students

also in their future academic or industrial carriers.

Raibulet C. and Ovaska E..

Hints for Organizing a Successful Doctoral Symposium - An Experience Report.

DOI: 10.5220/0004762800890094

In Proceedings of the 6th International Conference on Computer Supported Education (CSEDU-2014), pages 89-94

ISBN: 978-989-758-021-5

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

1.2 Which Are the Lessons Learned?

Concerning the organization work, the co-chairs

have learned that there is little material (except the

Web sites of the conferences and workshops) on the

organization rules of Doctoral Symposiums in

general, and on the evaluation criteria in particular.

Concerning the feedback for the PhD students,

the co-chairs have to outline that PhD students are

often confused and have difficulties to present a

research agenda for three-four years. Especially,

describing the expected results and their validation

seems particularly difficult for students. However,

they are rather familiar with presenting workshop

and conference papers which have a different

objective. They also have difficulties to interact with

domain experts (potential stakeholders) due to

various reasons.

This paper summarizes the experience of

organizing a Doctoral Symposium from the co-

chairs point of view. The paper addresses aspects

concerning both organization aspects and lessons to

be learnt by the PhD students.

The rest of the paper is organized as follows.

Section 2 presents the main steps of organizing a

Doctoral Symposium, as well as the motivation

behind these steps. Section 3 describes an actual

example by focusing on the evaluation of the

contributions. The lessons to be learnt are listed in

Section 4. Conclusions are dealt with in Section 5.

2 ORGANIZATIONAL HINTS

This section lists the meaningful aspects the co-

chairs have to address during the organization of the

Doctoral Symposium.

2.1 Call for Contributions

The call for contributions should indicate the macro

research topics of the Doctoral Symposium. Usually,

they are the same or a sub-set of the topics of the

main conference which the symposium is co-located

with. This aspect is important because it is closely

related to the expertise of the program committee

members. Typically, part of the main conference

program committee members also serve as the

evaluators of PhD students research plans and are

available and committed to face discussions with

PhD students.

The call for contributions should mention the

minimum and maximum length of the expected

papers. The length may be imposed by the editors of

the proceedings if the papers are accepted to be

published in the conference proceedings or in a co-

related publication. If no constraints are imposed,

the contributions should be long enough to present

the main aspects of the students' work, and in the

same time as short as possible in order to avoid the

insertion of details.

The call for contributions should clearly state

what main aspects are to be addressed in the

contribution and further used in the evaluation

process. It is fair that the students know in advance

the rules of the game. Based on their experience and

after making a survey on the call for contributions

for Doctoral Symposiums co-located in main

software engineering conferences (e.g., ICSE (ICSE,

2013), FSE (FSE, 2013), ASE (ASE, 2013),

ASWEC (ASWEC, 2013), APSEC (APSEC, 2013),

WICSA (WICSA, 2013)) the co-chairs consider as

significant the following ones:

 the problem to be solved, its location and

importance in the research field; this shows the

ability of the PhD students to focus on a

problem, to locate it in the research field and to

evaluate the possible impact if the results are

achieved;

 previous work, which has addressed similar

problems explaining why they have not been

previously solved; this shows the ability of the

PhD students to understand the research field and

to avoid already known problems and mistakes;

 the proposed approach; this shows the ability of

the PhD students to find appropriate solutions to

problems and to make a medium-long term plan

for achieving the identified solutions;

 the expected results; this shows the ability of the

students to identify the impact and the value of

their work, as well as the expected results after

investing a significant effort to solve a non-

trivial problem;

 a plan for the evaluation of the results; this

shows the ability of the PhD students to sustain

and demonstrate concretely the obtained results;

this request is usually consciously or

unconsciously avoided by the students, hence it

has been decided to explicitly introduce it in the

call for contributions.

Another significant aspect mentioned in the call for

contribution is the stage of the PhD work. This

influences the evaluation process (see Section 2.3).

2.2 Program Committee

The program committee should fulfil at least three

CSEDU2014-6thInternationalConferenceonComputerSupportedEducation

different criteria:

 members should be known experts in the field;

 the committee should be balanced and include

members from academia and industry;

 members (or at least part of them) should be

present during the workshop to discuss directly

with the students; this is not easily achievable

also because of the parallel events (e.g., sessions,

workshops) usually organized during

conferences, and hence experts may be involved

in various events contemporaneously.

2.3 Evaluation Criteria

Based on the current stage of the PhD work, the five

main points listed in the call for contributions are

translated into the following evaluation criteria:

 problem addressed and its link to the Doctoral

Symposium topics;

 motivation of the problem as an open research

issue;

 importance of the problem in the research field;

 identification of the main related works;

 a proposed research plan with milestones for

evaluation;

 appropriateness of the research plan for the

problem;

 expected results;

 evaluation plan of the expected results;

 appropriateness of the evaluation plan for the

results.

Furthermore, the clearness and the presentation

quality are added to this list of evaluation criteria.

These evaluation criteria should be made

available to the reviewers to achieve a homogeneous

evaluation and as objective as possible results.

The co-chairs decided to not accept contributions

which are in a very early stage (less than 6 months)

because it is difficult for the students to present

properly the last three points listed in the evaluation

criteria. In early stage, students may start to know

the previous work and try to identify a real problem

and formulate it in a meaningful way. However, it is

typical that they are not able to define a research

plan, identify the possible results and/or make their

evaluation plans. Furthermore, the co-chairs decided

to not accept contributions which are in the

advanced stage (less than 6 months to their finish)

because the symposium date is usually a couple of

months after the submission of the contributions and

the feedback provided by the program committee

members may hardly influence the thesis even if

problems or open issues are identified.

The program committee members' feedback and

the impact of the feedback on the students work are

different in an early stage and in an advanced stage.

In an early stage, the feedback may influence

significantly the students' work starting from the

definition of the problem until the evaluation plan of

the results. In an advanced stage, the feedback may

still influence the proposed approach and may focus

on the results and their evaluation plan.

2.4 Letter of Recommendation

The role of the letter of recommendation is to

confirm that the submitter is currently a PhD

student. In addition, it should indicate the current

stage of the PhD student's work and provide a

qualitative and brief evaluation of the work done

until the submission of the contribution.

2.5 Presentations and Feedback

The students' presentations during the PhD

workshop should be perceived by the students as a

short version of their final dissertation presentation.

For example, a solution may be to allocate 45' to

each student, 25' minutes for the presentation and 20

minutes for discussion. The discussion should

consider both positive and negative aspects of the

presented work in a constructive way.

2.6 Extended Abstract for the Doctoral

Symposium

The co-chairs have to prepare an extended abstract

about the Doctoral Symposium, abstract which is

included in the conference proceedings. The

extended abstract summarizes the call for

contributions, introduces and thanks the members of

the program committee for their collaboration, and

provides an overview of the accepted papers

(because the Doctoral Symposium papers are not

always included in the main conference

proceedings) (Ovaska, 2010).

3 AN ACTUAL EXAMPLE

In the case of this Doctoral Symposium, the received

contributions were eleven, among which seven have

been accepted for the presentation during the

Doctoral Symposium. Three of accepted papers got

an "accept" decision, while four of them a "weak

accept" one.

HintsforOrganizingaSuccessfulDoctoralSymposium-AnExperienceReport

The strong points of the "accepted" papers can be

summarized as follows:

(1) real open problems identified,

(2) clear presentation of the problem and of the plan

to address it,

(3) meaningful case study presentations,

(4) proper identification of the expected results,

(5) sustainable evaluation plan of the results.

However, also for the accepted papers improvement

aspects have been suggested:

(1) extend the evaluation plan to further case studies

or application domains (because some of them

seemed narrow-scoped),

(2) discuss the impact of the results both from

theoretical and practical points of view,

(3) delimit better the contribution of the PhD work if

it is developed in the context of a research

group/project which involves many researchers.

The strong points of the "weak accepted" papers can

be summarized as follows:

(1) meaningful research question identified,

(2) potentially good solution plan introduced,

(3) acceptable expected results,

(4) promising evaluation of the expected results.

The limitations of the "weak accepted" papers

concern:

(1) the unclearness of the overall presentation,

(2) poorly or very briefly addressed some of the

required points explicitly indicated in the call for

contributions,

(3) missing details for the comprehension of the

overall approach,

(4) English-language presentation problems (which

may contribute to the low quality related to the

first three mentioned problems),

(5) format presentation problems (e.g., the requested

format has not been adopted and hence, the

length of the contribution is less or longer than

the requested one).

The main reasons why four of the papers have been

rejected are the following. One paper was not

focused on the conference topics (e.g., the main

stream of this software engineering conference

research topics) and hence, on the topics of the co-

located Doctoral Symposium. The second rejected

paper was in a very early stage and the evaluation

criteria were not met. In spite of the fact that the

problem addressed and the idea behind the solution

were very challenging, the paper was rejected. The

third rejected paper did not describe clearly the

addressed problem, the proposed approach, and the

expected results. Its presentation quality was also

quite poor. The fourth rejected paper was a

particular case because it was a short version of the

paper submitted and accepted to be presented during

the main conference. In addition, it did not address

all the requirements specified in the call for

contributions, being thought from the beginning for

the main conference.

The qualitative evaluation (considering as

values: excellent, good, fair, and poor) of the

contributions is shown in Figure 1.

Figure 1: Example of evaluation results concerning the actual example.

CSEDU2014-6thInternationalConferenceonComputerSupportedEducation

4 LESSONS LEARNED

This section summarizes the main lessons the

organizers and the PhD students should learn from

and for a Doctoral Symposium.

4.1 Lessons to Be Learned by the PhD

Students: Preparation of

Contributions

PhD students interested in presenting their work in

the context of a Doctoral Symposium should first

understand the following hints:

1. Read carefully the call for contributions. First,

meet the topics and then try to address all the

requirements.

2. A Doctoral Symposium paper is not just another

paper. PhD students should be really motivated

and interested to receive a valuable feedback

from experts to improve their three-four years

work. It should describe a research plan/agenda,

which may address one or more challenging

research problems connected among them and

which should be developed through three-four

years. It is true, three-four years may seem a very

long period especially in the IT world, and this

may lead to changes in the agenda. However, the

PhD students should be able to provide and

sustain their research agenda also in a changing

research environment.

3. The PhD research may be co-located within one

or more research projects. An important

observation should be made here: the projects

should not be confused with the research

plan/agenda of a PhD student. Students should

clearly state their own agenda and the link with

the co-related projects. In addition, a PhD

student should clearly delimit his/her own

contribution from the other members of the

research group.

4. Do not try to invent problems. Try to find a real

one, also considering the interaction with the

industrial partners and lessons learned in industry

settings.

5. The page limit may be a problem. Try to be

concise, and in the same time precise.

4.2 Lessons to Be Learned by the PhD

Students: Presentations and

Feedback

The PhD students who have their work accepted to

be presented at a Doctoral Symposium should

consider the following hints:

1. Each presentation should indicate the potential

application domains of the results of the research

work and describe how the results will be

validated in a case study or a set of case studies.

Even if in the paper there is no space for such

information, the presentation should include

application domains and case studies.

2. PhD students should listen carefully to the

questions and the comments made by the

audience (program committee members and

other experts) and only after the interlocutor has

finished the question/feedback try to answer and

provide further details. PhD students tend to

answer immediately showing that they have

quickly understood the intention of the

interlocutor and that they have already thought

about the raised problem. Students should admit

that the audience may know more on a particular

topic than themselves, even if there are persons

in the audience who do not publish papers in this

domain, but work in this domain.

3. PhD students should consider their interlocutors

as friends, not enemies, which try to help them

by even making uncomfortable questions and

comments. Furthermore, they should establish

closer contacts with (at least part of) the program

committee members.

4. PhD students should also listen carefully to the

other PhD students' presentations and ask

questions about their work because they can

learn about the more advanced PhD students’

work and avoid repeating the similar problems in

their own work. Furthermore, the presenters will

be their future colleagues.

4.3 Lessons to Be Learned by the

Doctoral Symposium Co-chairs

The organizers of a Doctoral Symposium should

consider the following aspects:

1. Insert a request in the call for contributions

which asks for a gantt concerning the main

milestones for the proposed plan/agenda. Or,

require the presentation of the gantt during the

Doctoral Symposium.

2. The letter of recommendation has currently a

formal role, indicating the submitter is currently

a PhD student and that the research field is the

one described in the paper. In addition, it

specifies the year of the PhD research. This letter

should have a more important role. It should

HintsforOrganizingaSuccessfulDoctoralSymposium-AnExperienceReport

provide an overview of the student research and

organizational abilities and a qualitative

evaluation of the current status of the PhD

student’s work. It should also state the

collaborators in this work (e.g., MsC students

making their final thesis on topics concerning the

research plan, as well as senior researchers in

case the PhD work is co-located with an

European or National project).

3. Require a list the accepted and submitted

publications concerning the PhD research plan.

4. Provide a template for presenting the PhD

research plans and clearly stipulate that it should

be strictly followed.

5. Encourage PhD students’ participation by

allocating them one of the papers for evaluation.

In this way, a PhD student can practice his/her

competence on making analysis and representing

the evaluation results in a critical, but polite

manner and discuss about them with the program

committee and workshop members.

5. Be always two or more co-chairs especially if the

organizational period (e.g., review process) may

include holidays and one of the co-chairs may

not be available for several days.

5 CONCLUSIONS

PhD students are the talent of the future who shall

solve the emerging challenges in increased complex

situations in innovative ways. Thus, it is extremely

important that they can get all the support for

making their journey towards the core of their

research community as fruitful and easy as possible.

The primary intend of this paper is to support PhD

students in making top-class research and to provide

the guidelines for the chairs of Doctoral

Symposiums to organize successful events.

Indirectly, this kind of guidance can influence the

quality and effectiveness of research, which in turn

will have a positive impact on IT based innovation

and business as well.

ACKNOWLEDGEMENTS

We would like to thank the conference organizers

for giving us the opportunity to chair this event, and

the program committee members for their valuable

collaboration.

REFERENCES

Almi, N. E. A. M., Rahman, N. A., Purusothaman, D.,

Sulaiman, S., 2011. Software Engineering Education:

The Gap Between Industry's Requirements and

Graduates' Readiness. In IEEE Symposium on

Computers and Informatics, pp. 542-547.

APSEC Series, 2013. Asia-Pacific Software Engineering

Conference. http://apsec2013.eng.chula.ac.th/.

ASE Series, 2013. IEEE/ACM International Conference

on Automated Software Engineering.

http://ase2014.org/.

ASWEC Series, 2013. Australian Software Engineering

Conference", http://www.aswec2014.org/.

ECSA Series, 2013. European Conference on Software

Architectures. http://ecsa2014.cs.univie.ac.at/.

FSE Series, 2013. ACM SIGSOFT International

Conference on Foundations of Software Engineering.

http://fse22.gatech.edu/.

ICSE Series, 2013. International Conference on Software

Engineering. http://2014.icse-conferences.org/.

Ovaska, E., Raibulet, C., 2010. Doctoral Symposium of

the European Conference on Software Architecture

2010. In Proceedings of the Fourth European

Conference on Software Architecture: Companion

Volume, ACM International Conference Proceedings

Series, pp. 1-3.

Subrahmanyam, G. V. B., 2009. A Dynamic Framework

for Software Engineering Education Curriculum to

Reduce the Gap between Software Organizations and

Software Educational Institutes. In Proceedings on

Software Engineering Education and Training, pp.

248-254.

Villavicencio, M., Abran, A., 2012. The Necessary

Software Measurement knowledge in Software

Engineering Education from the Practitioners' Point of

View. In Proceedings of the IEEE Canadian

Conference on Electrical and Computer Engineering,

pp. 1-5.

WICSA Series, 2013. Working IEEE/IFIP Conference on

Software Architecture. http://wicsa2014.org/.

Xueyun, J., Zihui, C., 2010. A New Way to Software

Engineering Education. In Proceedings of the

International Conference on Educational and

Information Technology, pp. 1-3.

CSEDU2014-6thInternationalConferenceonComputerSupportedEducation