Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grammar: add module summary (introduced in LLVM 7.0) #43

Open
mewmew opened this issue Nov 24, 2018 · 5 comments
Open

grammar: add module summary (introduced in LLVM 7.0) #43

mewmew opened this issue Nov 24, 2018 · 5 comments

Comments

@mewmew
Copy link
Member

mewmew commented Nov 24, 2018

In LLVM 7.0 the concept of ThinLTO module summaries was introduced. We currently have no IR representation of module summaries, and the grammar for module summaries has not yet been written.

A test case containing a module summary is present in llvm/test/Assembler/thinlto-summary.ll:

; ModuleID = 'thinlto-summary.thinlto.bc'

^0 = module: (path: "thinlto-summary1.o", hash: (1369602428, 2747878711, 259090915, 2507395659, 1141468049))
^1 = module: (path: "thinlto-summary2.o", hash: (2998369023, 4283347029, 1195487472, 2757298015, 1852134156))

; Check a function that makes several calls with various profile hotness, and a
; reference (also tests forward references to function and variables in calls
; and refs).
^2 = gv: (guid: 1, summaries: (function: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 10, calls: ((callee: ^15, hotness: hot), (callee: ^17, hotness: cold), (callee: ^16, hotness: none)), refs: (^13))))

; Function with a call that has relative block frequency instead of profile
; hotness.
^3 = gv: (guid: 2, summaries: (function: (module: ^1, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 10, calls: ((callee: ^15, relbf: 256)))))

; Summaries with different linkage types.
^4 = gv: (guid: 3, summaries: (function: (module: ^0, flags: (linkage: internal, notEligibleToImport: 0, live: 0, dsoLocal: 1), insts: 1)))
; Make this one an alias with a forward reference to aliasee.
^5 = gv: (guid: 4, summaries: (alias: (module: ^0, flags: (linkage: private, notEligibleToImport: 0, live: 0, dsoLocal: 1), aliasee: ^14)))
^6 = gv: (guid: 5, summaries: (function: (module: ^0, flags: (linkage: available_externally, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 1)))
^7 = gv: (guid: 6, summaries: (function: (module: ^0, flags: (linkage: linkonce, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 1)))
^8 = gv: (guid: 7, summaries: (function: (module: ^0, flags: (linkage: linkonce_odr, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 1)))
^9 = gv: (guid: 8, summaries: (function: (module: ^0, flags: (linkage: weak_odr, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 1)))
^10 = gv: (guid: 9, summaries: (function: (module: ^0, flags: (linkage: weak, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 1)))
^11 = gv: (guid: 10, summaries: (variable: (module: ^0, flags: (linkage: common, notEligibleToImport: 0, live: 0, dsoLocal: 0))))
; Test appending globel variable with reference (tests backward reference on
; refs).
^12 = gv: (guid: 11, summaries: (variable: (module: ^0, flags: (linkage: appending, notEligibleToImport: 0, live: 0, dsoLocal: 0), refs: (^4))))

; Test a referenced global variable.
^13 = gv: (guid: 12, summaries: (variable: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0))))

; Test a dsoLocal variable.
^14 = gv: (guid: 13, summaries: (variable: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 1))))

; Functions with various flag combinations (notEligibleToImport, Live,
; combinations of optional function flags).
^15 = gv: (guid: 14, summaries: (function: (module: ^1, flags: (linkage: external, notEligibleToImport: 1, live: 1, dsoLocal: 0), insts: 1)))
^16 = gv: (guid: 15, summaries: (function: (module: ^1, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 1, funcFlags: (readNone: 1, noRecurse: 1))))
; This one also tests backwards reference in calls.
^17 = gv: (guid: 16, summaries: (function: (module: ^1, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 1, funcFlags: (readOnly: 1, returnDoesNotAlias: 1), calls: ((callee: ^15)))))

; Alias summary with backwards reference to aliasee.
^18 = gv: (guid: 17, summaries: (alias: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 1), aliasee: ^14)))

; Test all types of TypeIdInfo on function summaries.
^19 = gv: (guid: 18, summaries: (function: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 4, typeIdInfo: (typeTests: (^24, ^26)))))
^20 = gv: (guid: 19, summaries: (function: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 8, typeIdInfo: (typeTestAssumeVCalls: (vFuncId: (^27, offset: 16))))))
^21 = gv: (guid: 20, summaries: (function: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 5, typeIdInfo: (typeCheckedLoadVCalls: (vFuncId: (^25, offset: 16))))))
^22 = gv: (guid: 21, summaries: (function: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 15, typeIdInfo: (typeTestAssumeConstVCalls: (vFuncId: (^27, offset: 16), args: (42), vFuncId: (^27, offset: 24), args: (43))))))
^23 = gv: (guid: 22, summaries: (function: (module: ^0, flags: (linkage: external, notEligibleToImport: 0, live: 0, dsoLocal: 0), insts: 5, typeIdInfo: (typeCheckedLoadConstVCalls: (vFuncId: (^28, offset: 16), args: (42))))))

; Test TypeId summaries:

; Test the AllOnes resolution, and all kinds of WholeProgramDevirtResolution
; types, including all optional resolution by argument kinds.
^24 = typeid: (name: "_ZTS1A", summary: (typeTestRes: (kind: allOnes, sizeM1BitWidth: 7), wpdResolutions: ((offset: 0, wpdRes: (kind: branchFunnel)), (offset: 8, wpdRes: (kind: singleImpl, singleImplName: "_ZN1A1nEi")), (offset: 16, wpdRes: (kind: indir, resByArg: (args: (1, 2), byArg: (kind: indir, byte: 2, bit: 3), args: (3), byArg: (kind: uniformRetVal, info: 1), args: (4), byArg: (kind: uniqueRetVal, info: 1), args: (5), byArg: (kind: virtualConstProp)))))))
; Test TypeId with other optional fields (alignLog2/sizeM1/bitMask/inlineBits)
^25 = typeid: (name: "_ZTS1B", summary: (typeTestRes: (kind: inline, sizeM1BitWidth: 0, alignLog2: 1, sizeM1: 2, bitMask: 3, inlineBits: 4)))
; Test the other kinds of type test resoultions
^26 = typeid: (name: "_ZTS1C", summary: (typeTestRes: (kind: single, sizeM1BitWidth: 0)))
^27 = typeid: (name: "_ZTS1D", summary: (typeTestRes: (kind: byteArray, sizeM1BitWidth: 0)))
^28 = typeid: (name: "_ZTS1E", summary: (typeTestRes: (kind: unsat, sizeM1BitWidth: 0)))
@mewmew mewmew changed the title Add IR representation of module summary and support for parsing Add IR representation of ThinLTO module summary and support for parsing Nov 24, 2018
@mewmew
Copy link
Member Author

mewmew commented Nov 26, 2018

I'll mark this for a later release. I have yet to find an LLVM IR file containing ThinLTO module summary information outside of the test case suite.

@mewmew mewmew added this to the v0.4 milestone Nov 26, 2018
@mewmew
Copy link
Member Author

mewmew commented Nov 26, 2018

Also adding the help wanted tag, should anyone feel like taking a stab at implementing this :)

@mewmew mewmew modified the milestones: v0.4, future Nov 26, 2018
@mewmew
Copy link
Member Author

mewmew commented Oct 28, 2019

The 8.0 release of LLVM introduced the notion of GlobalVarSummary, along with the new varFlags keyword.

As of current, the LLVM IR grammar in llir/ll does not have support for global variables summaries. Any help with updating the grammar to add support for this would be warmly appreciated.

To the brave soul who is up for this challenge, I can recommend downloading the source release of LLVM for version 7.0 and 8.0 and running diff on the lib/AsmParser.

$ diff -r -u llvm-7.0.0.src/lib/AsmParser/ llvm-8.0.0.src/lib/AsmParser/
+++ llvm-8.0.0.src/lib/AsmParser/LLParser.cpp	2019-10-24 23:46:14.584517952 +0200

+/// GVarFlags
+///   ::= 'varFlags' ':' '(' 'readonly' ':' Flag ')'
+bool LLParser::ParseGVarFlags(GlobalVarSummary::GVarFlags &GVarFlags) {
+  assert(Lex.getKind() == lltok::kw_varFlags);
+  Lex.Lex();
+
+  unsigned Flag;
+  if (ParseToken(lltok::colon, "expected ':' here") ||
+      ParseToken(lltok::lparen, "expected '(' here") ||
+      ParseToken(lltok::kw_readonly, "expected 'readonly' here") ||
+      ParseToken(lltok::colon, "expected ':' here"))
+    return true;
+
+  ParseFlag(Flag);
+  GVarFlags.ReadOnly = Flag;
+
+  if (ParseToken(lltok::rparen, "expected ')' here"))
+    return true;
+  return false;
+}
+

@mewmew mewmew changed the title Add IR representation of ThinLTO module summary and support for parsing update grammar to LLVM 8.0: Add IR representation of ThinLTO module summary and support for parsing Oct 28, 2019
@mewmew mewmew mentioned this issue Oct 28, 2019
16 tasks
@mewmew mewmew added the grammar label Oct 28, 2019
@mewmew mewmew changed the title update grammar to LLVM 8.0: Add IR representation of ThinLTO module summary and support for parsing update grammar to LLVM 8.0: add module summary Nov 1, 2019
@mewmew
Copy link
Member Author

mewmew commented Nov 1, 2019

For anyone interested in implementing support for module summaries, take a look lib/AsmParser/LLParser.cpp.

In particular, the following functions are relevant:

  • bool LLParser::ParseGVEntry
  • LLParser::ParseFunctionSummary
  • LLParser::ParseVariableSummary
  • LLParser::ParseAliasSummary
  • LLParser::ParseOptionalVTableFuncs
  • LLParser::ParseOptionalRefs
  • LLParser::ParseOptionalTypeIdInfo
  • LLParser::ParseTypeTests
  • LLParser::ParseVFuncIdList
  • LLParser::ParseConstVCallList
  • LLParser::ParseConstVCall
  • LLParser::ParseGVFlags(GlobalValueSummary
  • LLParser::ParseGVarFlags
  • LLParser::ParseModuleReference
  • LLParser::ParseGVReference

Grammar extracted from comments:

/// ParseGVEntry
///   ::= 'gv' ':' '(' ('name' ':' STRINGCONSTANT | 'guid' ':' UInt64)
///         [',' 'summaries' ':' Summary[',' Summary]* ]? ')'
/// Summary ::= '(' (FunctionSummary | VariableSummary | AliasSummary) ')'

/// FunctionSummary
///   ::= 'function' ':' '(' 'module' ':' ModuleReference ',' GVFlags
///         ',' 'insts' ':' UInt32 [',' OptionalFFlags]? [',' OptionalCalls]?
///         [',' OptionalTypeIdInfo]? [',' OptionalRefs]? ')'

/// VariableSummary
///   ::= 'variable' ':' '(' 'module' ':' ModuleReference ',' GVFlags
///         [',' OptionalRefs]? ')'

/// AliasSummary
///   ::= 'alias' ':' '(' 'module' ':' ModuleReference ',' GVFlags ','
///         'aliasee' ':' GVReference ')'

/// OptionalVTableFuncs
///   := 'vTableFuncs' ':' '(' VTableFunc [',' VTableFunc]* ')'
/// VTableFunc ::= '(' 'virtFunc' ':' GVReference ',' 'offset' ':' UInt64 ')'

/// OptionalRefs
///   := 'refs' ':' '(' GVReference [',' GVReference]* ')'

/// OptionalTypeIdInfo
///   := 'typeidinfo' ':' '(' [',' TypeTests]? [',' TypeTestAssumeVCalls]?
///         [',' TypeCheckedLoadVCalls]?  [',' TypeTestAssumeConstVCalls]?
///         [',' TypeCheckedLoadConstVCalls]? ')'

/// TypeTests
///   ::= 'typeTests' ':' '(' (SummaryID | UInt64)
///         [',' (SummaryID | UInt64)]* ')'

/// VFuncIdList
///   ::= Kind ':' '(' VFuncId [',' VFuncId]* ')'

/// ConstVCallList
///   ::= Kind ':' '(' ConstVCall [',' ConstVCall]* ')'

/// ConstVCall
///   ::= '(' VFuncId ',' Args ')'

/// GVFlags
///   ::= 'flags' ':' '(' 'linkage' ':' OptionalLinkageAux ','
///         'notEligibleToImport' ':' Flag ',' 'live' ':' Flag ','
///         'dsoLocal' ':' Flag ',' 'canAutoHide' ':' Flag ')'

/// GVarFlags
///   ::= 'varFlags' ':' '(' 'readonly' ':' Flag
///                      ',' 'writeonly' ':' Flag ')'

/// ModuleReference
///   ::= 'module' ':' UInt

/// GVReference
///   ::= SummaryID

@mewmew mewmew changed the title update grammar to LLVM 8.0: add module summary grammar: add module summary (introduced in LLVM 8.0) Nov 1, 2019
@mewmew mewmew changed the title grammar: add module summary (introduced in LLVM 8.0) grammar: add module summary (introduced in LLVM 7.0) Nov 1, 2019
@mewmew
Copy link
Member Author

mewmew commented Dec 16, 2019

Relevant test cases from #111 (comment)

Grammar related to Module Summary

test cases failing related to Module Summary grammar
  • llvm/test/Assembler/thinlto-summary.ll
  • llvm/test/Assembler/thinlto-vtable-summary.ll
  • llvm/test/LTO/X86/Inputs/dllimport.ll
  • llvm/test/LTO/X86/dllimport.ll
  • llvm/test/Assembler/asm-path-writer.ll
    • syntax error at line 5

mewmew added a commit that referenced this issue Aug 1, 2022
Also, remove Module Summary diffs (tracked by #43).
mewmew added a commit that referenced this issue Aug 1, 2022
Also, remove Module Summary diffs (tracked by #43).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant