This webpage lists a set of suggestions for writing papers that are likely to be appreciated by the reviewers. The motivation for this page is that many times good papers fail to be appreciated (and hence accepted) due to misunderstandings between reviewers and authors. We hope that this document will help improve the average quality of the submitted papers.
Some rules are mandatory (e.g., the page limits and the double-blind submission rules) and if the paper does not comply with these, it will be rejected and may not even be reviewed. Other points are simply suggestions for organizing and structuring the paper that make it easier for reviewers to appreciate the quality of the work that has been done.
This document is in the form of an FAQ. You can use it as a checklist and go through it while writing your paper and before submission. Remember that you can re-submit updated versions of your paper until the deadline. So, if you realize there is something wrong just after you clicked on the “submit” button, you can still update the submitted paper until the deadline expires.
Scope: Does the paper have to focus on hard real-time systems?
First, in Track 1, papers are in scope even if they do not address timing issues as long as they pertain to embedded systems (broadly construed) and satisfy all track requirements (i.e., report on an actual system).
Second, while Tracks 2 and Tracks 3 require the presented work to pertain to time-sensitive systems, this class of systems extends far beyond classical (hard) real-time systems and generally also includes systems with implicit or soft timing expectations, such as Cloud and IoT systems characterized by tail-latency expectations or service-level agreements (SLAs).
Page limits: is there a page limit? Is there a specific format for submissions?
Yes; please see the submission instructions. In short, the paper must be in the same format as used in the final published proceedings, i.e. two columns, 10pt text. Word and Latex templates for formatting your paper can be found here. (Note: the conference uses US Letter (8.5″ x 11″) trim size).
Papers are limited to a maximum of 10 pages of technical content plus an optional appendix of supplementary material of up to 2 pages. Additional pages are permitted for the bibliography and acknowledgments only. The main body of the paper (10 pages) must be self-contained, as the reviewers’ decisions on whether to accept or reject it will be made on the basis of the information provided in those first 10 pages. The optional 2 pages appendix must only be used for supplementary material. Papers exceeding these page limits will be rejected and may not even be reviewed.
Please help the reviewers by putting page numbers on the submitted version of the paper. In LaTeX, this can be achieved by adding the following:
after \maketitle. If your paper is accepted, then do not forget to remove the page numbers again in the camera ready version.
If you submit a paper with an appendix and the paper is accepted, then you are required to include this material in the camera-ready version and to pay the extra page fee for each page beyond the 10th. In this case, after the paper has been accepted, for the camera-ready version you may re-organize the supplementary material as you see fit, i.e., it does not necessarily have to be presented as an appendix. Note this only applies after acceptance and not to the initial submission, whose technical content (excluding bibliography and acknowledgments) is strictly limited to a maximum of 10 pages plus the optional 2 pages appendix.
What about the appendix? What is supplementary material?
Good examples of supplementary material include:
- additional experimental results for other case studies, other benchmarks, or other scenarios that complement those already contained in the main body of the paper,
- proofs for some reasonably intuitive lemmas that are used as building blocks in the more substantial proofs of one or more theorems given in the main body of the paper,
- extra self-contained results (analytical or experimental) that complement those given in the main body of the paper, but are an aside to the main contribution of the paper.
Poor examples of supplementary material, highly likely to result in the paper being rejected include:
- proofs for the key theoretical results in the paper – in this case the main body of the paper is not self-contained and its correctness cannot be judged without the appendix,
- all of the experimental results – RTAS papers must contain a discussion of the experimental configuration and results in the main body of the paper,
- the discussion of related work – the main body of the paper must be self-contained and put the research presented into context.
Material in the appendix may be read at the reviewer’s discretion. If on reading the main body of the paper, a reviewer’s recommendation is to accept the paper, then they opt to also check the appendix for correctness. If their recommendation is reject, then they may choose not to read the supplementary material in the appendix.
What if there still isn’t enough space?
The general idea is that the submitted version of a paper must contain the same material that will, in the case of acceptance, be refined following the reviewer’s comment and go into the final proceedings. If you have additional material that you want to make available to the reviewers that will not fit in the appendix, then you must:
- write a technical report, that can be of any length and any format, containing the additional material;
- anonymize the report to make it suitable for double-blind review;
- submit the technical report as part of the submission process (see double-blind submission requirements); and
- reference the technical report in the submitted paper, stating that it is available upon request from the track chair.
After the paper is accepted:
- make the technical report (with author names and affiliations) available on an open-access repository, such as your institution repository (if available), arXiv, or other similar long-term repositories; and
- reference the technical report in the camera-ready paper, giving a URL and a tech report number, so that readers know where to obtain it.
Note that the additional material in the technical report will not be published in the proceedings, and will only be read at the reviewers’ discretion. Remember that reviewers must evaluate your paper and not the technical report. If the reviewers think the paper contains insufficient material to be published, then the paper may be rejected, regardless of the contents of the technical report. Reviewers are not obliged to read material in a technical report, so it may or may not be examined, even if the paper is accepted.
Originality: How do I convince the reviewers that my work is actually new?
The paper should clearly state the research problem, together with information about the key contributions. A good example of this is to have a clear explanation of the problem in the introduction, together with a well-balanced positioning of why this problem has not been completely solved before. A poor example is to just describe an implementation without saying anything about why this implementation exists or which problem it actually tries to solve. Without this information, the reviewers are left guessing what the actual problem is, which may lead to significant misunderstandings.
You should aim to clearly explain the relationship between your work and the existing state-of-the-art in terms of related work. Make clear where you have built on existing results and the key differences between the proposed approach and those that have been published previously, including prior work of your own. Also consider adding a short paragraph in the conclusions where you briefly summarize the main innovative technical contributions of the paper.
As well as explaining the advantages of your new approach with respect to the state-of-the-art, also give a balanced view of its disadvantages. This is generally well appreciated by reviewers, and preferable to leaving it to the reviewers to point out any shortcomings.
Please, remember that double submissions are unacceptable because they could compromise the requirement for originality of the paper, and effectively duplicate the review effort. Check that your paper has enough new material with respect to other papers you published or submitted to other conferences/journals. Including material from a 4 or 6 pages work-in-progress or workshop paper that has been published is acceptable, provided that it does not have a DOI, or at least 30% new content has been added. Nevertheless, you must disclose the existence of the prior paper at the time of submission (see double-blind submission requirements). After acceptance, you should include a citation to that preliminary publication and explain its relationship to the paper.
Presentation: How important is writing quality?
The quality of the writing in the paper is very important. The main goal of a paper is to communicate the technical aspects of the work to other people. If the paper is not written in correct English, reviewers may not be able to understand the merits of the paper. Therefore, it is more likely that your paper will get a low score on the technical aspects as well as on the presentation. Take some time to review the presentation of your paper. Automated spelling and grammar checkers can help avoid many minor mistakes. As well as proof reading your paper a number of times yourself, it is recommended that you ask other good writers among your colleagues for their comments and opinions on it. The submitted paper must already be correct. After it has been reviewed is not the time to make corrections to the writing.
If you struggle with English as a second language, you can substantially improve the chance of acceptance by retaining the services of a professional copy-editing service, of which there exist many, to correct the presentation of your paper (grammar, style) once the content is complete.
Where to define notation and acronyms?
Make sure that any notation used is clearly defined and distinct (do not use symbols that can easily be confused with one another). Over the years, a de-facto standard notation has developed that is used in many real-time systems papers. Where possible try and stay with this common notation as it gives reviewers familiar with it far fewer new things to remember when they read your paper. The best place to define terminology and notation is together in a section called “system model, terminology and notation” or something similar. This gives reviewers a single place to refer back to where they can find any symbols they need to look up again. If you have a large amount of notation in your paper, consider providing a table of notation. Make sure you define all notation and acronyms before they are used.
How to format figures?
Make sure that all of your figures, diagrams and graphs are legible when printed out in black and white. (This is the way many reviewers still look at the papers and most likely the way they will appear in the printed proceedings). Avoid the temptation to make your figures the size of a postage stamp or thumb nail in order to fit your content into the page limits. Figures should normally fill the column width. Make sure the different lines on the graphs are clearly distinguishable by using markers that are obviously different and where necessary using different line types (e.g. dashed, dotted). Make sure the text on the graphs is legible and not too small.
When using figures, make sure you use vector graphics (rather than bitmaps) whenever possible as vector graphics will render much better if reviewers (and later readers) prefer to read the PDF on a digital device with a zoomable high-DPI interface (e.g., an iPad).
Evaluation: Is it mandatory to have experimental evaluations in the paper?
Yes, all RTAS papers are required to contain a section on experimental results. These may arise from the implementation of a real system, or via the use of data from case studies, benchmarks or models of real systems. The use of synthetic workloads or models is permitted if well motivated. Note, the different tracks have different requirements for experimental evaluation (see the track descriptions for details).
The basic idea is that a reader of your paper should be able to reproduce your experiments and obtain the same results. Hence, it is necessary to describe the experimental setup, including details of case study or benchmark data (or where it can be obtained) and how synthetic data (if used) has been generated. If you are reporting statistical data, then make sure you present measures pertaining to the quality of the results obtained, for example confidence intervals, or variance.
To aid in the reproducibility of results, consider also making your evaluation code available by uploading it onto your web page or institution’s repository and including a reference and URL to it in the final paper. Note this is not compulsory but is well appreciated by the community. Note that, at the time of submission, supplemental materials (such as code) should be provided in blinded form and not directly linked (see double-blind submission requirements).
Authors are expected to compare their work with the existing state-to-the-art to show the size, scope and significance of the improvements they have made. Experimental evaluations should seek to cover as broad but realistic range of parameters and cases as possible, giving a balanced view of performance, where possible with respect to existing work. Authors should avoid selecting specific cases where there may be a bias in favour of their approach.
This FAQ has been a joint effort by the RTAS and ECRTS communities, a living document started by Giuseppe Lipari for ECRTS 2006 with contributions by many others since.