Version date: Nov 2021
UCL DEPARTMENT OF MEDICAL PHYSICS AND BIOMEDICAL ENGINEERING
MPHY0035 MSc Research Project / MPHY0034 MRes Research Project
Final Report Assessment Form
Name of Student: Yuxiang Jia
Name of Assessor: Haroon Chughtai
(Tick box): First Supervisor Second Supervisor
Marking Guidance
Each supervisor must produce an independent report.
If the two supervisors’ marks differ by more that 10% points, moderation may be required.
For MSc students, the Final Report counts for 60% of the Research Projects module.
For MSc students, the expectation is the Final Report will consist of between 8,000 to 10,000 words.
For MRes students, the Final Report counts for 45% of the Research Projects module.
For MRes students, the expectation is the Final Report will consist of between 15,000 to 20,000 words.
The pass mark is 50%
Completed report forms should be emailed to [email protected]
The table below provides some examples of descriptions that would apply to the specified marking levels
across the five marking categories:
Descriptors Scientific
context
Project
planning &
management
Research
progress &
contribution
Limitations &
conclusions
Clarity and
presentation
40% – Poor
The student
provides a
(descriptor) review
of relevant
literature, and
demonstrates a
(descriptor)
understanding of
the research field
and the wider
context of the
project throughout
the report.
The student
demonstrates
(descriptor) project
planning and time
management. They
also demonstrate a
(descriptor) ability
to adapt their
research approach
in response to new
results or
circumstances
Relative to
reasonable
expectations, the
student
demonstrates
(descriptor)
progress in their
research project.
Their report
demonstrates they
have made a
(descriptor)
individual
contribution to the
research area.
The student shows
a (descriptor)
awareness and
understanding of
the limitations of
their methods. The
student provides a
(descriptor)
description of
improvements that
could be made to
their research. Any
conclusions drawn
are justified to a
(descriptor) degree.
The student has
provided a
(descriptor) level of
detail. The spelling,
grammar, syntax,
clarity of writing,
tables, figures and
referencing are
mostly (descriptor).
50% – Satisfactory
60% – Good
70% – Strong
80% – Excellent
Final Report
Total Mark (%)
Mark: 44
Comments
(please provide a detailed justification of each of the marks that should be entered on the following page):
Scientific Context (SC)
Overall, the literature review contained some relevant literature, but this was limited and not always completely
relevant to the problem being explored in the project. There was a satisfactory understanding of the research
field and the wider context of the project throughout the report.
The initial problem statement did not concisely state the wider context of project and the problem being
tackled. The context around increases in medical data and the potential vulnerabilities in hospital IT networks
was not clearly linked to the challenges in learning from data held in multiple organisations. There was a
divergence into the risks of commercial reselling of medical data, but the relevance of reducing risks to patient
privacy was not clearly linked to it.
However, the general idea of the use of federated learning to allow patient data from many sources to be used
to improve medical-related machine learning, whilst reducing the risks to patient rights and privacy, eventually
came through.
A large section of the background focussed on medical data sharing platforms, but it was unclear why there
was such a focus on this as it focussed on the sharing of medical records for direct patient care where
identifiable data sharing is needed. Contrary to this, there was no exploration of existing federated learning
platforms or other approaches for medical imaging-related tasks. Whilst there is evidence that the student
understands that federated learning can be used to reduce risks around sharing sensitive data, it appears
there is a lack of understanding of what scenarios it would be most useful in.
There appears to be some understanding of federated learning and machine learning concepts, but these are
not explained in the report in depth so are difficult to assess. For example, little critical explanation is given as
to why a U-Net model was a good initial choice for segmentation tasks, and what it’s limitations are.
Project Planning and Management (PP)
The aims and outcomes of the project are stated in an appropriate section of the report. There is evidence of
reflection on why these aims, and outcomes are appropriate in relation the scientific context of the report, but
the commentary is largely superficial and has not evolved since the interim report.
The methods section does not appear to relate back to the stated aims and objectives, suggesting a lack of
forward planning or adapting as circumstances changed.
Overall, the student demonstrated poor project planning and time management during the project.
Research Progress and Contribution (RP)
The research progress presented in th
e report shows the modification of PyTorch based Federated Learning
code with additional datasets, models and evaluation metrics in order to perform a number of experiments
assessing the performance of a federated learning approach.
More detail about the implementation process and how the work was achieved should have been provided
(e.g. a code listing or link to hosted code).
For the measurement metrics described in 3. Methods, sufficient explanation of their use and relevance in the
context of the experiments undertaken was not explained. An overall rationale for the overall experimental
design is not explained in the report, but the individual stages are stated and explained in some detail.
Results are poorly presented and are not clearly interpreted. They should have been explained in the report in
more detail with the figures explicitly referenced. In many cases conclusions are made without explaining the
rationale or citing appropriate theory in the literature.
Version date: Nov 2021
Overall, there was minimal scientific contribution to the design and implementation of the methods used, and
some data collected showing a comparison between “baseline” and federated learning approaches for a
number of experiments conducted.
Limitation and Conclusions (LC)
There is some discussion on the challenges of medical images, and the difficulty of segmentation tasks in that
context. It is noted that the number of data sets available is always a limiting factor. However, the conclusion
that the Federated Learning strategy is effective is reached without presenting a concise argument.
There is a poor linking back to prior work and a heavy emphasis on the development of medical data sharing
platforms, without explicitly considering what the results from the report mean for the field.
Key points around the benefits of federated learning strategies are stated in the conclusion. The main future
work identified is again the need for larger datasets in machine learning tasks. However no mention is made to
objectives presented in the project plan about stable and reproducible results. Further work around improving
the experimental setup to be more representative of real-world scenarios would have been desirable.
Clarity and Presentation (CP)
Overall, the report has a coherent structure in terms of paragraphs and sections. However, some sections are
not clearly introduced or could have made more sense in other parts of the report. For example, 3. Methods
goes straight into measurement metrics without setting any context for the overall experimental design, as well
as introducing a different model to that introduced in the literature review and background.
Grammar, punctuation, and spelling is mainly correct throughout the report. However, the language used
alternates between more formal and more colloquial, with many incorrect or undefined definitions of key terms
and acronyms. In later sections of the report (e.g. 3. Methods), the language becomes increasingly less formal.
There is a lack of continuity between ideas which severely disrupts the flow of arguments. More concise
explanation of ideas with clear links between them would have greatly improved the structure of the report.
This results in a writing style that is sometimes difficult to follow with several paragraphs that are poorly
structured and make the report difficult to follow in places.
There is a large block of repeated text in section 2.2 – more proof-reading would have identified this.
Throughout the report, there are numerous instances where statements are presented without satisfactory
detail or supporting references.
In several cases the references provided link to low quality sources that do not provide information on where
the contained information originally came from.
In several cases, the cited sources do not support the statement written (e.g. Konecny, Jakub et al.
“Federated Learning: Strategies for Improving Communication Efficiency.” ArXiv abs/1610.05492
(2016) is about communication efficiency in federated machine learning settings, and not as cited in
the report a leading paper in introducing federated learning as a concept.)
Beyond 3.5 Data Collection, no references are present, aside from hyperlinked text without an accompanying
bibliography entry.
Additional figures including diagrams and graphs would have been helpful in explaining key concepts to the
reader. For example, the types of federated learning strategies could have been explained with the use of
diagrams.
Where figures are included, they aren’t consistently cross-referenced in the report, or explained in satisfactory
detail in captions. Where figures were sourced from third parties, they should have been referenced or credited
appropriately
Version date: Nov 2021
Version date: Nov 2021
Please add your marks for each component in the boxes below
1. Scientific context Mark (/20)
9
2. Project planning and management Mark (/20)
8
3. Research progress & contribution Mark (/30)
13
4. Limitations & conclusions Mark (/20)
9
4. Clarity and presentation Mark (/10)
5
TOTAL Mark (/100)
44