Transformer with Tree-order Encoding for Neural Program Generation

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearch

Authors

While a considerable amount of semantic parsing approaches have employed RNN architectures for code generation tasks, there have been only few attempts to investigate the applicability of Transformers for this task. Including hierarchical information of the underlying programming language syntax has proven to be effective for code generation. Since the positional encoding of the Transformer can only represent positions in a flat sequence, we have extended the encoding scheme to allow the attention mechanism to also attend over hierarchical positions in the input. Furthermore, we have realized a decoder based on a restrictive grammar graph model to improve the generation accuracy and ensure the well-formedness of the generated code. While we did not surpass the state of the art, our findings suggest that employing a tree-based positional encoding in combination with a shared natural-language subword vocabulary improves generation performance over sequential positional encodings.
Original languageEnglish
Title of host publicationConference XXX
Number of pages10
DOIs
Publication statusIn preparation - 30.05.2022
Externally publishedYes

Bibliographical note

This paper was authored in late 2020 and early 2021 for the most part

    Research areas

  • cs.CL, cs.AI, 68T07, 68T50, I.2.7
  • Informatics

Recently viewed

Publications

  1. Applying Bayesian Parameter Estimation to A/B Tests in e-Business Applications
  2. Preventive Emergency Detection Based on the Probabilistic Evaluation of Distributed, Embedded Sensor Networks
  3. How to combine collaboration scripts and heuristic worked examples to foster mathematical argumentation - when working memory matters
  4. Enabling Road Condition Monitoring with an on-board Vehicle Sensor Setup
  5. Gain Scheduling Controller for Improving Level Control Performance
  6. Control of the inverse pendulum based on sliding mode and model predictive control
  7. Clustering Hydrological Homogeneous Regions and Neural Network Based Index Flood Estimation for Ungauged Catchments
  8. Spatial mislocalization as a consequence of sequential coding of stimuli
  9. Analysis of long-term statistical data of cobalt flows in the EU
  10. Building Assistance Systems using Distributed Knowledge Representations
  11. Model inversion using fuzzy neural network with boosting of the solution
  12. From entity to process
  13. Topic Embeddings – A New Approach to Classify Very Short Documents Based on Predefined Topics
  14. Tree diversity increases forest temperature buffering via enhancing canopy density and structural diversity
  15. Evaluating structural and compositional canopy characteristics to predict the light-demand signature of the forest understorey in mixed, semi-natural temperate forests
  16. lp-Norm Multiple Kernel Learning
  17. Design optimization of spiral coils for textile applications by genetic algorithm
  18. Exact and approximate inference for annotating graphs with structural SVMs
  19. Recurrence Quantification Analysis of Processes and Products of Discourse
  20. Lessons learned for spatial modelling of ecosystem services in support of ecosystem accounting
  21. Clause identification using entropy guided transformation learning
  22. Mathematical Modeling for Robot 3D Laser Scanning in Complete Darkness Environments to Advance Pipeline Inspection
  23. An analytical approach to evaluating nonmonotonic functions of fuzzy numbers
  24. An analytical predictor machine learning corrector scheme for modeling lateral flow in hot strip rolling
  25. Improving students’ science text comprehension through metacognitive self-regulation when applying learning strategies
  26. “Ideation is Fine, but Execution is Key”
  27. Comments on "Tracking Control of Robotic Manipulators With Uncertain Kinematics and Dynamics"
  28. From Knowledge to Application
  29. Neural correlates of the enactment effect in the brain
  30. How Much Tracking Is Necessary? - The Learning Curve in Bayesian User Journey Analysis
  31. Data based analysis of order processing strategies to support the positioning between conflicting economic and logistic objectives
  32. Optimization of 3D laser scanning speed by use of combined variable step
  33. Machine Learning and Knowledge Discovery in Databases
  34. Modelling biodegradability based on OECD 301D data for the design of mineralising ionic liquids
  35. Efficient Order Picking Methods in Robotic Mobile Fulfillment Systems
  36. Towards Advanced Learning in Dispatching Rule-Based Scheuling
  37. Learning and Re-learning from net- based cooperative learning discourses
  38. Using Heider’s Epistemology of Thing and Medium for Unpacking the Conception of Documents: Gantt Charts and Boundary Objects
  39. Conceptual understanding of complex components and Nyquist-Shannon sampling theorem