Mirrors_Framework/ghidra
1
0
Fork 0
mirror of https://github.com/NationalSecurityAgency/ghidra.git synced 2026-06-04 21:30:22 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

GP-6643 Added ability to order Libraries via an ordinal assignment. This ordered list is now used by external symbol resolver analyzer.

Browse Source
This commit is contained in:
ghidra1
2026-04-20 09:27:11 -04:00
parent 7be1d8c943
commit 5e82e2869c
32 changed files with 1473 additions and 553 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Ghidra/Features/Base/src/main/help/help/topics/ReferencesPlugin/external_program_names.htm
+91 -46
View File
@@ -14,14 +14,36 @@
<BODY lang="EN-US"> <BODY lang="EN-US">
<H1><A name="View_External_Program_Names"></A>External Program Names</H1> <H1><A name="View_External_Program_Names"></A>External Program Names</H1>
<P>An external reference is a reference to a location in another program. The reference <P>An external location identifies an external function or data dependency
destination includes the name of some program. To use an external reference to navigate, the associated with a named Library (i.e., External Program). One or more external references may refer to
external program name must be associated with an existing program file in the Ghidra project. a single external location. Thunk functions within a Program may also refer to an external
<A name="Resolve"></A>If the association has been defined, then the external reference is said location if it corresponds to a function. Each Library defined within Ghidra can optionally
to have been <B><I>resolved</I></B>. The <I>External Programs</I> view manages the associations be associated with another Program file within the Ghidra project. </P>
between external program names and program files. The table shows all external program names
and their associated links to program files. Use the <I>External Programs</I> view to add <P>There is a reserved
external names, delete external names, set associations, and clear associations.</P> Library named <B>&lt;EXTERNAL&gt;</B> which is a holding area for external locations whose
associated Library is unknown (commonly used by ELF Imports).
The <B>External Symbol Resolver</B> analyzer can be used to search
for these external locations among the ordered list of Libraries which have a Program file association.
The ordered Library sequence dictates the order that this analyzer searches through them.</P>
<P>To navigate on an external reference/location, or resolve external locations, the
external Library name must be associated with an existing Program file in the same project.
<A name="Resolve"></A>If a Library's Program association has been specified, then any related
external reference is said to have been <B><I>resolved</I></B>. </P>
<P>The <I>External Programs</I> view manages the associations
between external Library names and Program files as well as the ordered Library sequence.
Use the <I>External Programs</I> view to add/delete external Library names,
set/clear Program associations and adjust search order using up/down placement actions.
Other than the up/down ordering functionality, the
<A href="../SymbolTreePlugin/SymbolTree.htm#Externals">Symbol Tree</A> Imports node provides
similar actions plus the ability to navigate.</P>
<P><I><IMG src="help/shared/note.png" border="0"> The term "External Program" within Ghidra
is used interchangeably with the term "Library" and represents a dependency. Each defined
Library has a corresponding Library Symbol within a program's symbol table and may have related
external locations.</P>
<H2><A name="ExternalNamesDialog"></A>External Programs View</H2> <H2><A name="ExternalNamesDialog"></A>External Programs View</H2>
@@ -29,26 +51,33 @@
</P> </P>
<BLOCKQUOTE> <BLOCKQUOTE>
<P>The <I>External Programs</I> view consists of a main scrollable list of external program <P>The <I>External Programs</I> view consists of a main scrollable list of external Library
names and their associated Ghidra program files.</P> names and their associated Ghidra program files.</P>
<BLOCKQUOTE> <BLOCKQUOTE>
<H3>Name Column</H3> <H3>Name Column</H3>
<BLOCKQUOTE> <BLOCKQUOTE>
<P>The name of the external program. Many external programs will share the same external <P>The name of the external Library. Many external function and/or data locations will
program name. Setting or changing the associated Ghidra file will affect all the external correspond to the same external Library within a Ghidra program which are considered to
references to that name. Double-click on this field to edit the name. After you change be Imports.</P>
the name, hit the &lt;Enter&gt; key.</P>
<P>The name used within the current program to identify the Library may be
changed. To do this, double-click on the Name to enter edit mode. After you change
the name, press &lt;Enter&gt; to commit the change.</P>
</BLOCKQUOTE> </BLOCKQUOTE>
<H3>Ghidra Program Column<BR> <H3>Ghidra Program Column<BR>
</H3> </H3>
<BLOCKQUOTE> <BLOCKQUOTE>
<P>The Ghidra file associated with the external program name. This field is blank if <P>The Ghidra program file associated with the external Library name. This field is blank if
external reference has not been resolved. Ghidra will not be able to "follow" an external the external Library has not been resolved. Ghidra will not be able to "follow" an external
reference if its external program name does not have an associated Ghidra file.</P> location reference into a Library, or search for external symbols within a Library,
if a Library does not have a Ghidra program file association.</P>
<P>See <A href="#ChooseExternalProgram">Set External Path Association</A> for changing the
path shown.</P>
</BLOCKQUOTE> </BLOCKQUOTE>
<H3><A name="Add_External_Program_Name"></A>Add Button</H3> <H3><A name="Add_External_Program_Name"></A>Add Button</H3>
@@ -59,13 +88,47 @@
</P> </P>
</BLOCKQUOTE> </BLOCKQUOTE>
<H3><A name="Delete_External_Program_Name"></A>Delete External Name Button</H3>
<BLOCKQUOTE>
<P>The <SPAN style="font-weight: bold;">Delete</SPAN> <IMG alt="" src=
"images/edit-delete.png"> button deletes the selected external Library names from the
program.&nbsp; If a selected external Library name has associated external locations, it can
not be deleted. The <SPAN style="font-weight: bold;">Delete</SPAN> button is enabled
whenever one or more rows are selected.</P>
</BLOCKQUOTE>
<H3><A name="Move_Library_Up"></A>Move Library Up Button</H3>
<BLOCKQUOTE>
<P>The <SPAN style="font-weight: bold;">Up</SPAN> <IMG alt="" src=
"images/up.png"> button will shift a selected external Library up
within the list of Libraries thus increasing its priority when used to
search for external symbols. The <SPAN style="font-weight: bold;">Up</SPAN>
button is enabled whenever one external program name is selected and can be moved
up within the Library list.</P>
<P>
</BLOCKQUOTE>
<H3><A name="Move_Library_Down"></A>Move Library Down Button</H3>
<BLOCKQUOTE>
<P>The <SPAN style="font-weight: bold;">Down</SPAN> <IMG alt="" src=
"images/down.png"> button will shift a selected external Library down
within the list of Libraries thus reducing its priority when used to
search for external symbols. The <SPAN style="font-weight: bold;">Down</SPAN>
button is enabled whenever one external program name is selected and can be moved
down within the Library list.</P>
<P>
</BLOCKQUOTE>
<H3><A name="ChooseExternalProgram"></A><A name="Set_External_Name_Association"></A>Set <H3><A name="ChooseExternalProgram"></A><A name="Set_External_Name_Association"></A>Set
External Name Association Button</H3> External Path Association Button</H3>
<BLOCKQUOTE> <BLOCKQUOTE>
<P>The <SPAN style="font-weight: bold;">Set</SPAN> <IMG alt="" src= <P>The <SPAN style="font-weight: bold;">Set</SPAN> <IMG alt="" src=
"images/editbytes.gif"> button brings up a <I>Ghidra program chooser</I> dialog. Choose a "images/editbytes.gif"> button brings up a <I>Ghidra program chooser</I> dialog. Choose a
Ghidra program file to associate it with the selected external program name. This button Ghidra program file to associate it with the selected external Library name. This button
is only enabled when a single external program name is selected.</P> is only enabled when a single external program name is selected.</P>
</BLOCKQUOTE> </BLOCKQUOTE>
</BLOCKQUOTE> </BLOCKQUOTE>
@@ -77,26 +140,13 @@
<BLOCKQUOTE> <BLOCKQUOTE>
<BLOCKQUOTE> <BLOCKQUOTE>
<H3><A name="Clear_External_Name_Association"></A>Clear External Name Association <H3><A name="Clear_External_Name_Association"></A>Clear External Path Association
Button</H3> Button</H3>
<BLOCKQUOTE> <BLOCKQUOTE>
<P>This <SPAN style="font-weight: bold;">Clear <IMG alt="" src= <P>The <SPAN style="font-weight: bold;">Clear <IMG alt="" src=
"images/edit-delete.png">&nbsp;</SPAN> button clears the assocated program for all the "Icons.CLEAR_ICON">&nbsp;</SPAN> button clears the associated program path for all the
selected external program names.<BR> selected rows.<BR>
</P>
</BLOCKQUOTE>
<H3><A name="Delete_External_Program_Name"></A>Delete External Name Button</H3>
<BLOCKQUOTE>
<P>This <SPAN style="font-weight: bold;">Delete</SPAN> <IMG alt="" src=
"images/edit-delete.png"> button deletes the selected external program names from the
program.&nbsp; If a selected external program name contains external locations, it can
not be deleted. The <SPAN style="font-weight: bold;">Delete</SPAN> button is enabled
whenever one or more external program names are selected.</P>
<P><BR>
</P> </P>
</BLOCKQUOTE> </BLOCKQUOTE>
</BLOCKQUOTE> </BLOCKQUOTE>
@@ -115,17 +165,12 @@
<LI>Enter the new external program name into the pop-up dialog.</LI> <LI>Enter the new external program name into the pop-up dialog.</LI>
<LI style="list-style: none"> <P><I><IMG src="help/shared/note.png" border="0">Any new External Program added
&lt;&gt; will be placed at the bottom of the list. If relying on external symbol resolution
analysis and the library search order is important, the new entry may be moved up
in the list using the Up button.
</I></P>
<P><I><IMG src="help/shared/note.png" border="0"> If the table is sorted by Name, then
the name you enter will be placed at the correct position in the table to maintain the
sort order. The sort icon <IMG src="Icons.SORT_ASCENDING_ICON" border="0"> or <IMG
src="Icons.SORT_DESCENDING_ICON" border="0"> indicates the order and what column is
being sorted. You can also sort by Ghidra Pathname by clicking on the header for this
column. Click on the sort icon to change the order.</I></P>
&lt;/&gt;
</LI>
</OL> </OL>
<H3>Resolving an External Name to an existing Ghidra program<BR> <H3>Resolving an External Name to an existing Ghidra program<BR>
@@ -157,7 +202,7 @@
<LI>Click on the external program name that has an association to be cleared.</LI> <LI>Click on the external program name that has an association to be cleared.</LI>
<LI>Press the <B>Clear</B> <IMG alt="" src="images/erase16.png"> button</LI> <LI>Press the <B>Clear</B> <IMG alt="" src="Icons.CLEAR_ICON"> button</LI>
</OL> </OL>
<H3>Removing an External Program Name</H3> <H3>Removing an External Program Name</H3>
Ghidra/Features/Base/src/main/help/help/topics/ReferencesPlugin/images/External_names_dialog.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 8.9 KiB

After

Width:  |  Height:  |  Size: 11 KiB

Ghidra/Features/Base/src/main/java/ghidra/app/cmd/refs/ClearExternalNameCmd.java → Ghidra/Features/Base/src/main/java/ghidra/app/cmd/refs/ClearExternalPathCmd.java
+19 -10
View File
@@ -16,37 +16,46 @@
package ghidra.app.cmd.refs; package ghidra.app.cmd.refs;
import ghidra.framework.cmd.Command; import ghidra.framework.cmd.Command;
import ghidra.program.model.listing.Library;
import ghidra.program.model.listing.Program; import ghidra.program.model.listing.Program;
import ghidra.util.exception.AssertException; import ghidra.program.model.symbol.ExternalManager;
import ghidra.util.exception.InvalidInputException; import ghidra.util.exception.InvalidInputException;
/** /**
* Command to remove an external program name from the reference manager. * Command to clear the external program path associated with an external Library.
* *
*/ */
public class ClearExternalNameCmd implements Command<Program> { public class ClearExternalPathCmd implements Command<Program> {
private String externalName; private String externalName;
private String status; private String status;
private boolean userDefined = true; private boolean userDefined = true;
/** /**
* Constructs a new command removing an external program name. * Constructs a new command for clearing the external program path associated with a
* @param externalName the name of the external program name to be removed. * specified external Library.
* @param externalName external Library name
*/ */
public ClearExternalNameCmd(String externalName) { public ClearExternalPathCmd(String externalName) {
this.externalName = externalName; this.externalName = externalName;
} }
@Override @Override
public boolean applyTo(Program program) { public boolean applyTo(Program program) {
try { try {
program.getExternalManager().setExternalPath(externalName, null, userDefined); // Avoid creating the Library if it does not already exist
ExternalManager externalManager = program.getExternalManager();
Library lib = externalManager.getExternalLibrary(externalName);
if (lib != null) {
externalManager.setExternalPath(externalName, null, userDefined);
return true;
}
status = "Library not found: " + externalName;
} }
catch (InvalidInputException e) { catch (InvalidInputException e) {
throw new AssertException(e); status = e.getMessage();
} }
return true; return false;
} }
@Override @Override
@@ -56,7 +65,7 @@ public class ClearExternalNameCmd implements Command<Program> {
@Override @Override
public String getName() { public String getName() {
return "Remove External Program Name"; return "Clear External Library Path";
} }
} }
Ghidra/Features/Base/src/main/java/ghidra/app/cmd/refs/SetExternalNameCmd.java
+33 -10
View File
@@ -16,7 +16,11 @@
package ghidra.app.cmd.refs; package ghidra.app.cmd.refs;
import ghidra.framework.cmd.Command; import ghidra.framework.cmd.Command;
import ghidra.program.model.listing.Library;
import ghidra.program.model.listing.Program; import ghidra.program.model.listing.Program;
import ghidra.program.model.symbol.ExternalManager;
import ghidra.program.model.symbol.SourceType;
import ghidra.util.exception.DuplicateNameException;
import ghidra.util.exception.InvalidInputException; import ghidra.util.exception.InvalidInputException;
/** /**
@@ -26,29 +30,48 @@ public class SetExternalNameCmd implements Command<Program> {
private String externalName; private String externalName;
private String externalPath; private String externalPath;
private SourceType source;
private String status; private String status;
private boolean userDefined = true;
/** /**
* Constructs a new command for setting the external program name and path. * Constructs a new command for creating a Library, if it does not exist, and optionally
* @param externalName the name of the link. * setting the associated external program path. If created, a {@link SourceType#USER_DEFINED}
* @param externalPath the path of the file to associate with this link. * source will be specified.
* @param externalName the Library name.
* @param externalPath the project file path of the program file to associate with the Library.
*/ */
public SetExternalNameCmd(String externalName, String externalPath) { public SetExternalNameCmd(String externalName, String externalPath) {
this(externalName, externalPath, SourceType.USER_DEFINED);
}
/**
* Constructs a new command for creating a Library, if it does not exist, and optionally
* setting the associated external program path.
* @param externalName the Library name.
* @param externalPath the project file path of the program file to associate with the Library.
* @param source the symbol source type to be applied if the library must be created.
*/
public SetExternalNameCmd(String externalName, String externalPath, SourceType source) {
this.externalName = externalName; this.externalName = externalName;
this.externalPath = externalPath; this.externalPath = externalPath;
this.source = source;
} }
@Override @Override
public boolean applyTo(Program program) { public boolean applyTo(Program program) {
try { try {
program.getExternalManager().setExternalPath(externalName, externalPath, userDefined); ExternalManager externalManager = program.getExternalManager();
Library library = externalManager.getExternalLibrary(externalName);
if (library == null) {
externalManager.addExternalLibraryName(externalName, source);
}
library.setAssociatedProgramPath(externalPath);
} }
catch (InvalidInputException e) { catch (DuplicateNameException | InvalidInputException e) {
status = "Invalid name specified"; status = e.getMessage();
return false;
} }
return true; return false;
} }
@Override @Override
@@ -58,7 +81,7 @@ public class SetExternalNameCmd implements Command<Program> {
@Override @Override
public String getName() { public String getName() {
return "Set External Program Name"; return "Set External Library Name and Path";
} }
} }
Ghidra/Features/Base/src/main/java/ghidra/app/merge/listing/ExternalProgramMerger.java
+36 -14
View File
@@ -26,8 +26,7 @@ import ghidra.app.merge.*;
import ghidra.app.merge.util.ConflictUtility; import ghidra.app.merge.util.ConflictUtility;
import ghidra.app.util.HelpTopics; import ghidra.app.util.HelpTopics;
import ghidra.program.model.address.Address; import ghidra.program.model.address.Address;
import ghidra.program.model.listing.Program; import ghidra.program.model.listing.*;
import ghidra.program.model.listing.ProgramChangeSet;
import ghidra.program.model.symbol.*; import ghidra.program.model.symbol.*;
import ghidra.program.util.ProgramMerge; import ghidra.program.util.ProgramMerge;
import ghidra.program.util.SimpleDiffUtility; import ghidra.program.util.SimpleDiffUtility;
@@ -448,20 +447,36 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
mergeManager.updateProgress(progress, mergeManager.updateProgress(progress,
"Merging external program information for " + name + "..."); "Merging external program information for " + name + "...");
String originalPath = Library originalLib =
(originalName != null) ? originalExtMgr.getExternalLibraryPath(originalName) : null; (originalName != null) ? originalExtMgr.getExternalLibrary(originalName) : null;
String latestPath = Library latestLib =
(latestName != null) ? latestExtMgr.getExternalLibraryPath(latestName) : null; (latestName != null) ? latestExtMgr.getExternalLibrary(latestName) : null;
String myPath = (myName != null) ? myExtMgr.getExternalLibraryPath(myName) : null; Library myLib = (myName != null) ? myExtMgr.getExternalLibrary(myName) : null;
if (same(latestName, myName) && same(latestPath, myPath)) {
String originalPath = (originalLib != null) ? originalLib.getAssociatedProgramPath() : null;
String latestPath = (latestLib != null) ? latestLib.getAssociatedProgramPath() : null;
String myPath = (myLib != null) ? myLib.getAssociatedProgramPath() : null;
int originalOrdinal =
originalLib != null ? originalExtMgr.getLibraryOrdinal(originalName) : -1;
int latestOrdinal = latestLib != null ? latestExtMgr.getLibraryOrdinal(latestName) : -1;
int myOrdinal = myLib != null ? myExtMgr.getLibraryOrdinal(myName) : -1;
if (same(latestName, myName) && same(latestPath, myPath) && latestOrdinal == myOrdinal) {
return; return;
} }
boolean changedLatestName = !same(originalName, latestName); boolean changedLatestName = !same(originalName, latestName);
boolean changedMyName = !same(originalName, myName); boolean changedMyName = !same(originalName, myName);
boolean changedLatestPath = !same(originalPath, latestPath); boolean changedLatestPath = !same(originalPath, latestPath);
boolean changedMyPath = !same(originalPath, myPath); boolean changedMyPath = !same(originalPath, myPath);
boolean changedLatest = changedLatestName || changedLatestPath;
boolean changedMy = changedMyName || changedMyPath; // Temper ordinal changes - don't allow them to drive a conflict but try to preserve my change.
// Ordinal changes may be lost during any library change conflict resolution.
boolean changedMyOrdinal = myLib != null && myOrdinal != originalOrdinal;
boolean changedLatest = changedLatestName || changedLatestPath; // ignore ordinal change
boolean changedMy = changedMyName || changedMyPath || changedMyOrdinal;
if (changedLatest) { if (changedLatest) {
if (changedMy) { if (changedMy) {
// conflict: Ask to keep latest or my // conflict: Ask to keep latest or my
@@ -473,7 +488,7 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
if (resultID != -1 && resultName == null) { if (resultID != -1 && resultName == null) {
resultName = latestName; // Need to create Library symbol in Result program. resultName = latestName; // Need to create Library symbol in Result program.
} }
autoMergeWhenOnlyLatestChanged(resultName, latestName, latestPath); autoMergeWhenOnlyLatestChanged(resultName, latestName, latestPath, latestOrdinal);
} }
} }
else { else {
@@ -491,13 +506,13 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
symbol.getSymbolType()); symbol.getSymbolType());
} }
} }
autoMergeWhenOnlyMyChanged(resultName, myName, myPath); autoMergeWhenOnlyMyChanged(resultName, myName, myPath, myOrdinal);
} }
} }
} }
private void autoMergeWhenOnlyLatestChanged(String resultName, String latestName, private void autoMergeWhenOnlyLatestChanged(String resultName, String latestName,
String latestPath) { String latestPath, int latestOrdinal) {
if (resultName == null) { if (resultName == null) {
// latestName appears to have been discarded during SymbolMerge. // latestName appears to have been discarded during SymbolMerge.
return; return;
@@ -509,6 +524,9 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
else { else {
resultExtMgr.setExternalPath(resultName, latestPath, resultExtMgr.setExternalPath(resultName, latestPath,
isExternalUserDefined(latestPgm, latestName)); isExternalUserDefined(latestPgm, latestName));
if (latestOrdinal >= 0) {
resultExtMgr.setLibraryOrdinal(resultName, latestOrdinal);
}
} }
} }
catch (InvalidInputException e) { catch (InvalidInputException e) {
@@ -516,7 +534,8 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
} }
} }
private void autoMergeWhenOnlyMyChanged(String resultName, String myName, String myPath) { private void autoMergeWhenOnlyMyChanged(String resultName, String myName, String myPath,
int myOrdinal) {
if (resultName == null) { if (resultName == null) {
// myName appears to have been discarded during SymbolMerge. // myName appears to have been discarded during SymbolMerge.
return; return;
@@ -528,6 +547,9 @@ public class ExternalProgramMerger implements MergeResolver, ListingMergeConstan
else { else {
resultExtMgr.setExternalPath(resultName, myPath, resultExtMgr.setExternalPath(resultName, myPath,
isExternalUserDefined(myPgm, myName)); isExternalUserDefined(myPgm, myName));
if (myOrdinal >= 0) {
resultExtMgr.setLibraryOrdinal(resultName, myOrdinal);
}
} }
} }
catch (InvalidInputException e) { catch (InvalidInputException e) {
Ghidra/Features/Base/src/main/java/ghidra/app/merge/listing/SymbolMerger.java
+2
View File
@@ -2463,6 +2463,8 @@ class SymbolMerger extends AbstractListingMerger {
ExternalManager srcExtMgr = srcPgm.getExternalManager(); ExternalManager srcExtMgr = srcPgm.getExternalManager();
String path = srcExtMgr.getExternalLibraryPath(name); String path = srcExtMgr.getExternalLibraryPath(name);
// NOTE: No attempt is made to preserve library ordinal
ExternalManagerDB extMgr = (ExternalManagerDB) resultPgm.getExternalManager(); ExternalManagerDB extMgr = (ExternalManagerDB) resultPgm.getExternalManager();
extMgr.setExternalPath(name, path, (source == SourceType.USER_DEFINED)); extMgr.setExternalPath(name, path, (source == SourceType.USER_DEFINED));
symbol = resultSymTab.getLibrarySymbol(name); symbol = resultSymTab.getLibrarySymbol(name);
Ghidra/Features/Base/src/main/java/ghidra/app/plugin/core/references/ExternalReferencesProvider.java
+65 -25
View File
@@ -27,7 +27,7 @@ import docking.ActionContext;
import docking.DefaultActionContext; import docking.DefaultActionContext;
import docking.action.builder.ActionBuilder; import docking.action.builder.ActionBuilder;
import docking.widgets.dialogs.InputDialog; import docking.widgets.dialogs.InputDialog;
import docking.widgets.table.AbstractSortedTableModel; import docking.widgets.table.AbstractGTableModel;
import generic.theme.GIcon; import generic.theme.GIcon;
import ghidra.app.cmd.refs.*; import ghidra.app.cmd.refs.*;
import ghidra.framework.cmd.Command; import ghidra.framework.cmd.Command;
@@ -51,6 +51,8 @@ import resources.Icons;
public class ExternalReferencesProvider extends ComponentProviderAdapter { public class ExternalReferencesProvider extends ComponentProviderAdapter {
private static Icon ADD_ICON = Icons.ADD_ICON; private static Icon ADD_ICON = Icons.ADD_ICON;
private static Icon DELETE_ICON = Icons.DELETE_ICON; private static Icon DELETE_ICON = Icons.DELETE_ICON;
private static Icon UP_ICON = Icons.UP_ICON;
private static Icon DOWN_ICON = Icons.DOWN_ICON;
private static Icon EDIT_ICON = new GIcon("icon.base.edit.bytes"); private static Icon EDIT_ICON = new GIcon("icon.base.edit.bytes");
private static Icon CLEAR_ICON = Icons.CLEAR_ICON; private static Icon CLEAR_ICON = Icons.CLEAR_ICON;
@@ -98,6 +100,22 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
.onAction(ac -> deleteExternalProgram()) .onAction(ac -> deleteExternalProgram())
.buildAndInstallLocal(this); .buildAndInstallLocal(this);
new ActionBuilder("Move Library Up", getOwner())
.popupMenuPath("Move Library Up")
.popupMenuIcon(UP_ICON)
.toolBarIcon(UP_ICON)
.enabledWhen(ac -> canDecrementSelectedLibraryOrdinal())
.onAction(ac -> adjustLibraryOrdinal(true))
.buildAndInstallLocal(this);
new ActionBuilder("Move Library Down", getOwner())
.popupMenuPath("Move Library Down")
.popupMenuIcon(DOWN_ICON)
.toolBarIcon(DOWN_ICON)
.enabledWhen(ac -> canIncrementSelectedLibraryOrdinal())
.onAction(ac -> adjustLibraryOrdinal(false))
.buildAndInstallLocal(this);
new ActionBuilder("Set External Name Association", getOwner()) new ActionBuilder("Set External Name Association", getOwner())
.popupMenuPath("Set External Name Association") .popupMenuPath("Set External Name Association")
.popupMenuIcon(EDIT_ICON) .popupMenuIcon(EDIT_ICON)
@@ -238,6 +256,48 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
} }
} }
private void adjustLibraryOrdinal(boolean moveUp) {
if ((moveUp && !canDecrementSelectedLibraryOrdinal()) ||
(!moveUp && !canIncrementSelectedLibraryOrdinal())) {
return;
}
int ordinalAdjustment = moveUp ? -1 : 1;
List<String> selectedExternalNames = getSelectedExternalNames();
String externalName = selectedExternalNames.get(0); // must be exactly one for us to be enabled.
ExternalManager externalManager = program.getExternalManager();
Library externalLibrary = externalManager.getExternalLibrary(externalName);
if (externalLibrary != null) {
program.withTransaction("Adjust Library Order", () -> {
int ordinal = externalManager.getLibraryOrdinal(externalName);
if (ordinal < 0) {
Msg.showError(this, mainPanel, "Library Ordering Change Error",
"Failed to update Library ordinal for: " + externalName);
return;
}
externalManager.setLibraryOrdinal(externalName, ordinal + ordinalAdjustment);
});
}
}
private boolean canDecrementSelectedLibraryOrdinal() {
int[] selectedRows = table.getSelectedRows();
if (selectedRows.length == 1 && selectedRows[0] != 0) {
ExternalNamesRow rowBefore = tableModel.getRowObject(selectedRows[0] - 1);
return !Library.UNKNOWN.equals(rowBefore.getName()); // cannot displace UNKNOWN Library
}
return false;
}
private boolean canIncrementSelectedLibraryOrdinal() {
int lastRow = table.getRowCount() - 1;
int[] selectedRows = table.getSelectedRows();
if (selectedRows.length == 1 && selectedRows[0] != lastRow) {
ExternalNamesRow row = tableModel.getRowObject(selectedRows[0]);
return !Library.UNKNOWN.equals(row.getName()); // cannot alter UNKNOWN Library ordinal
}
return false;
}
private void setExternalProgramAssociation() { private void setExternalProgramAssociation() {
List<String> selectedExternalNames = getSelectedExternalNames(); List<String> selectedExternalNames = getSelectedExternalNames();
String externalName = selectedExternalNames.get(0); // must be exactly one for us to be enabled. String externalName = selectedExternalNames.get(0); // must be exactly one for us to be enabled.
@@ -268,7 +328,7 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
private void clearExternalAssociation() { private void clearExternalAssociation() {
CompoundCmd<Program> cmd = new CompoundCmd<>("Clear External Program Associations"); CompoundCmd<Program> cmd = new CompoundCmd<>("Clear External Program Associations");
for (String externalName : getSelectedExternalNames()) { for (String externalName : getSelectedExternalNames()) {
cmd.add(new ClearExternalNameCmd(externalName)); cmd.add(new ClearExternalPathCmd(externalName));
} }
getTool().execute(cmd, program); getTool().execute(cmd, program);
} }
@@ -325,7 +385,7 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
} }
class ExternalNamesTableModel extends AbstractSortedTableModel<ExternalNamesRow> { class ExternalNamesTableModel extends AbstractGTableModel<ExternalNamesRow> {
final static int NAME_COL = 0; final static int NAME_COL = 0;
final static int PATH_COL = 1; final static int PATH_COL = 1;
@@ -336,18 +396,18 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
private List<ExternalNamesRow> paths = new ArrayList<>(); private List<ExternalNamesRow> paths = new ArrayList<>();
void updateTableData() { void updateTableData() {
paths.clear(); paths.clear();
if (program != null && isVisible()) { if (program != null && isVisible()) {
ExternalManager extMgr = program.getExternalManager(); ExternalManager extMgr = program.getExternalManager();
// NOTE: Keep table in ordinal sequence as provided by external manager
String[] programNames = extMgr.getExternalLibraryNames(); String[] programNames = extMgr.getExternalLibraryNames();
Arrays.sort(programNames);
for (String programName : programNames) { for (String programName : programNames) {
if (Library.UNKNOWN.equals(programName)) { if (Library.UNKNOWN.equals(programName)) {
continue; continue;
} }
ExternalNamesRow path = new ExternalNamesRow(programName, ExternalNamesRow path = new ExternalNamesRow(programName,
extMgr.getExternalLibraryPath(programName)); extMgr.getExternalLibraryPath(programName));
paths.add(path); paths.add(path);
@@ -391,11 +451,6 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
return "External Programs Model"; return "External Programs Model";
} }
@Override
public boolean isSortable(int columnIndex) {
return true;
}
@Override @Override
public boolean isCellEditable(int rowIndex, int columnIndex) { public boolean isCellEditable(int rowIndex, int columnIndex) {
if (columnIndex == NAME_COL) { if (columnIndex == NAME_COL) {
@@ -448,21 +503,6 @@ public class ExternalReferencesProvider extends ComponentProviderAdapter {
} }
} }
@Override
protected Comparator<ExternalNamesRow> createSortComparator(int columnIndex) {
if (columnIndex == PATH_COL) {
// force the path column to have a secondary compare using the name column
// to ensure a 'stable' sort. Without this during analysis
// the constant updates cause the table to sort randomly when
// there are lots of empty path values.
Comparator<ExternalNamesRow> c1 =
(r1, r2) -> Objects.requireNonNullElse(r1.getPath(), "")
.compareTo(Objects.requireNonNullElse(r2.getPath(), ""));
return c1.thenComparing((r1, r2) -> r1.getName().compareTo(r2.getName()));
}
return super.createSortComparator(columnIndex);
}
} }
} }
Ghidra/Features/Base/src/main/java/ghidra/app/plugin/core/symboltree/SymbolTreeProvider.java
+1 -2
View File
@@ -645,8 +645,7 @@ public class SymbolTreeProvider extends ComponentProviderAdapter {
.each(SYMBOL_REMOVED).call(this::processSymbolRemoved) .each(SYMBOL_REMOVED).call(this::processSymbolRemoved)
.each(EXTERNAL_ENTRY_ADDED, EXTERNAL_ENTRY_REMOVED) .each(EXTERNAL_ENTRY_ADDED, EXTERNAL_ENTRY_REMOVED)
.call(this::processExternalEntryChanged) .call(this::processExternalEntryChanged)
.any(EXTERNAL_PATH_CHANGED, EXTERNAL_NAME_ADDED, .any(EXTERNAL_NAME_ADDED, EXTERNAL_NAME_REMOVED, EXTERNAL_NAME_CHANGED)
EXTERNAL_NAME_REMOVED, EXTERNAL_NAME_CHANGED)
// Rather than try to find each affected symbol, it is easier to just reload // Rather than try to find each affected symbol, it is easier to just reload
// the tree. This is infrequent enough that it should not be disruptive. // the tree. This is infrequent enough that it should not be disruptive.
.call(this::reloadTree) .call(this::reloadTree)
Ghidra/Features/Base/src/main/java/ghidra/app/util/opinion/AbstractLibrarySupportLoader.java
+12
View File
@@ -39,6 +39,7 @@ import ghidra.program.model.listing.Library;
import ghidra.program.model.listing.Program; import ghidra.program.model.listing.Program;
import ghidra.program.model.symbol.ExternalManager; import ghidra.program.model.symbol.ExternalManager;
import ghidra.util.Msg; import ghidra.util.Msg;
import ghidra.util.StringUtilities;
import ghidra.util.exception.CancelledException; import ghidra.util.exception.CancelledException;
import ghidra.util.exception.InvalidInputException; import ghidra.util.exception.InvalidInputException;
import ghidra.util.task.TaskMonitor; import ghidra.util.task.TaskMonitor;
@@ -101,6 +102,17 @@ public abstract class AbstractLibrarySupportLoader extends AbstractProgramLoader
public static final String LOAD_ONLY_LIBRARIES_OPTION_NAME = "Only Load Libraries"; // hidden public static final String LOAD_ONLY_LIBRARIES_OPTION_NAME = "Only Load Libraries"; // hidden
static final boolean LOAD_ONLY_LIBRARIES_OPTION_DEFAULT = false; static final boolean LOAD_ONLY_LIBRARIES_OPTION_DEFAULT = false;
/**
* Gets a program property name to represent the ordered required library of the given index
*
* @param libraryIndex The index of the required library
* @return A program property name to represent the ordered required library of the given index
*/
public static String getRequiredLibraryProperty(int libraryIndex) {
return String.format("%s %s]", "Required Library [",
StringUtilities.pad("" + libraryIndex, ' ', 4));
}
/** /**
* Loads bytes in a particular format into the given {@link Program}. * Loads bytes in a particular format into the given {@link Program}.
* *
Ghidra/Features/Base/src/main/java/ghidra/app/util/opinion/ElfProgramBuilder.java
+2 -2
View File
@@ -54,7 +54,6 @@ import ghidra.program.model.scalar.Scalar;
import ghidra.program.model.symbol.*; import ghidra.program.model.symbol.*;
import ghidra.program.model.util.AddressSetPropertyMap; import ghidra.program.model.util.AddressSetPropertyMap;
import ghidra.program.model.util.CodeUnitInsertionException; import ghidra.program.model.util.CodeUnitInsertionException;
import ghidra.program.util.ExternalSymbolResolver;
import ghidra.util.*; import ghidra.util.*;
import ghidra.util.datastruct.*; import ghidra.util.datastruct.*;
import ghidra.util.exception.*; import ghidra.util.exception.*;
@@ -458,7 +457,8 @@ class ElfProgramBuilder extends MemorySectionResolver implements ElfLoadHelper {
String[] neededLibs = elf.getDynamicLibraryNames(); String[] neededLibs = elf.getDynamicLibraryNames();
for (String neededLib : neededLibs) { for (String neededLib : neededLibs) {
monitor.checkCancelled(); monitor.checkCancelled();
props.setString(ExternalSymbolResolver.getRequiredLibraryProperty(libraryIndex++), props.setString(
AbstractLibrarySupportLoader.getRequiredLibraryProperty(libraryIndex++),
neededLib); neededLib);
} }
} }
Ghidra/Features/Base/src/main/java/ghidra/app/util/opinion/MachoProgramBuilder.java
+1 -2
View File
@@ -57,7 +57,6 @@ import ghidra.program.model.reloc.RelocationResult;
import ghidra.program.model.reloc.RelocationTable; import ghidra.program.model.reloc.RelocationTable;
import ghidra.program.model.symbol.*; import ghidra.program.model.symbol.*;
import ghidra.program.model.util.CodeUnitInsertionException; import ghidra.program.model.util.CodeUnitInsertionException;
import ghidra.program.util.ExternalSymbolResolver;
import ghidra.util.*; import ghidra.util.*;
import ghidra.util.exception.*; import ghidra.util.exception.*;
import ghidra.util.task.TaskMonitor; import ghidra.util.task.TaskMonitor;
@@ -1298,7 +1297,7 @@ public class MachoProgramBuilder {
libraryPaths.add(libraryPath); libraryPaths.add(libraryPath);
addLibrary(libraryPath); addLibrary(libraryPath);
props.setString( props.setString(
ExternalSymbolResolver.getRequiredLibraryProperty(libraryIndex++), AbstractLibrarySupportLoader.getRequiredLibraryProperty(libraryIndex++),
libraryPath); libraryPath);
} }
} }
Ghidra/Features/Base/src/main/java/ghidra/program/util/ExternalSymbolResolver.java
+65 -102
View File
@@ -23,75 +23,18 @@ import java.util.function.Consumer;
import db.Transaction; import db.Transaction;
import ghidra.app.util.opinion.Loaded; import ghidra.app.util.opinion.Loaded;
import ghidra.framework.model.*; import ghidra.framework.model.*;
import ghidra.framework.options.Options;
import ghidra.program.model.listing.*; import ghidra.program.model.listing.*;
import ghidra.program.model.symbol.*; import ghidra.program.model.symbol.*;
import ghidra.util.Msg; import ghidra.util.Msg;
import ghidra.util.StringUtilities;
import ghidra.util.exception.*; import ghidra.util.exception.*;
import ghidra.util.task.TaskMonitor; import ghidra.util.task.TaskMonitor;
/** /**
* Moves dangling external function symbols found in the {@link Library#UNKNOWN EXTERNAL/UNKNOWN} * Moves dangling external function symbols found in the {@link Library#UNKNOWN EXTERNAL/UNKNOWN}
* namespace into the namespace of the external library that publishes a matching symbol. * namespace into the namespace of the external library that publishes a matching symbol.
* <p>
* This uses an ordered list of external library names that was attached to the program during
* import by the Elf or Macho loader (see {@link #REQUIRED_LIBRARY_PROPERTY_PREFIX}).
* *
*/ */
public class ExternalSymbolResolver implements Closeable { public class ExternalSymbolResolver implements Closeable {
private final static String REQUIRED_LIBRARY_PROPERTY_PREFIX = "Required Library [";
/**
* Gets a program property name to represent the ordered required library of the given index
*
* @param libraryIndex The index of the required library
* @return A program property name to represent the ordered required library of the given index
*/
public static String getRequiredLibraryProperty(int libraryIndex) {
return String.format("%s %s]", REQUIRED_LIBRARY_PROPERTY_PREFIX,
StringUtilities.pad("" + libraryIndex, ' ', 4));
}
/**
* Returns an ordered list of library names, as specified by the logic/rules of the original
* operating system's loader (eg. Elf / MachO dynamic library loading / symbol resolving
* rules)
*
* @param program The {@link Program}
* @return list of library names, in original order
*/
public static List<String> getOrderedRequiredLibraryNames(Program program) {
TreeMap<Integer, String> orderLibraryMap = new TreeMap<>();
Options options = program.getOptions(Program.PROGRAM_INFO);
for (String optionName : options.getOptionNames()) {
// Legacy programs may have the old "ELF Required Library [" program property, so
// we should not assume that the option name starts exactly with
// REQUIRED_LIBRARY_PROPERTY_PREFIX. We must deal with a potential substring at the
// start of the option name.
int prefixIndex = optionName.indexOf(REQUIRED_LIBRARY_PROPERTY_PREFIX);
if (prefixIndex == -1 || !optionName.endsWith("]")) {
continue;
}
String libName = options.getString(optionName, null);
if (libName == null) {
continue;
}
String indexStr = optionName
.substring(prefixIndex + REQUIRED_LIBRARY_PROPERTY_PREFIX.length(),
optionName.length() - 1)
.trim();
try {
orderLibraryMap.put(Integer.parseInt(indexStr), libName.trim());
}
catch (NumberFormatException e) {
Msg.error(ExternalSymbolResolver.class,
"Program contains invalid property: " + optionName);
}
}
return new ArrayList<>(orderLibraryMap.values());
}
private final ProjectData projectData; private final ProjectData projectData;
private final TaskMonitor monitor; private final TaskMonitor monitor;
@@ -236,33 +179,6 @@ public class ExternalSymbolResolver implements Closeable {
* Represents a program that needs its external symbols to be fixed. * Represents a program that needs its external symbols to be fixed.
*/ */
private class ProgramSymbolResolver { private class ProgramSymbolResolver {
record ExtLibInfo(String name, Library lib, String programPath, Program program,
List<String> resolvedSymbols, Throwable problem) {
String getProblemMessage() {
if (problem instanceof VersionException ve) {
return getVersionError(ve);
}
return problem != null ? problem.getMessage() : "";
}
String getLibPath() {
return programPath != null ? programPath : "missing";
}
String getVersionError(VersionException ve) {
String versionType = switch (ve.getVersionIndicator()) {
case VersionException.NEWER_VERSION -> " newer";
case VersionException.OLDER_VERSION -> "n older";
default -> "n unknown";
};
String upgradeMsg = ve.isUpgradable() ? " (upgrade is possible)" : "";
return "skipped: file was created with a%s version of Ghidra%s"
.formatted(versionType, upgradeMsg);
}
}
Program program; Program program;
String programPath; String programPath;
@@ -296,16 +212,19 @@ public class ExternalSymbolResolver implements Closeable {
logger.accept("\t%d external symbols resolved, %d remain unresolved" logger.accept("\t%d external symbols resolved, %d remain unresolved"
.formatted(getResolvedSymbolCount(), unresolvedExternalFunctionIds.size())); .formatted(getResolvedSymbolCount(), unresolvedExternalFunctionIds.size()));
for (ExtLibInfo extLib : extLibs) { for (ExtLibInfo extLib : extLibs) {
String libPath = extLib.getAssociatedProgramPath();
String loggedLibPath = libPath != null ? libPath : "missing";
if (extLib.problem != null) { if (extLib.problem != null) {
logger.accept("\t[%s] -> %s, %s".formatted(extLib.name, extLib.getLibPath(), logger.accept("\t[%s] -> %s, %s".formatted(extLib.getName(), loggedLibPath,
extLib.getProblemMessage())); extLib.getProblemMessage()));
} }
else if (extLib.programPath != null) { else if (libPath != null) {
logger.accept("\t[%s] -> %s, %d new symbols resolved".formatted(extLib.name, logger.accept(
extLib.getLibPath(), extLib.resolvedSymbols.size())); "\t[%s] -> %s, %d new symbols resolved".formatted(extLib.getName(),
loggedLibPath, extLib.resolvedSymbols.size()));
} }
else { else {
logger.accept("\t[%s] -> %s".formatted(extLib.name, extLib.getLibPath())); logger.accept("\t[%s] -> %s".formatted(extLib.getName(), loggedLibPath));
} }
if (!shortSummary) { if (!shortSummary) {
for (String symbolName : extLib.resolvedSymbols) { for (String symbolName : extLib.resolvedSymbols) {
@@ -328,8 +247,8 @@ public class ExternalSymbolResolver implements Closeable {
private boolean hasSomeLibrariesConfigured() { private boolean hasSomeLibrariesConfigured() {
for (ExtLibInfo extLib : extLibs) { for (ExtLibInfo extLib : extLibs) {
if (extLib.program != null || extLib.problem != null || if (extLib.problem != null ||
extLib.programPath != null) { extLib.getAssociatedProgramPath() != null) {
return true; return true;
} }
} }
@@ -367,15 +286,13 @@ public class ExternalSymbolResolver implements Closeable {
private List<ExtLibInfo> getLibsToSearch() throws CancelledException { private List<ExtLibInfo> getLibsToSearch() throws CancelledException {
List<ExtLibInfo> result = new ArrayList<>(); List<ExtLibInfo> result = new ArrayList<>();
ExternalManager externalManager = program.getExternalManager(); ExternalManager externalManager = program.getExternalManager();
for (String libName : getOrderedRequiredLibraryNames(program)) { // External manager supplies external Libraries in appropriate search order
Library lib = externalManager.getExternalLibrary(libName); for (Library lib : externalManager.getLibraries()) {
String libPath = lib != null ? lib.getAssociatedProgramPath() : null; String libPath = lib.getAssociatedProgramPath();
Program libProg = libPath != null ? getLibraryProgram(libPath) : null; Program libProg = libPath != null ? getLibraryProgram(libPath) : null;
Throwable problem = Throwable problem =
libProg == null && libPath != null ? problemLibraries.get(libPath) : null; libProg == null && libPath != null ? problemLibraries.get(libPath) : null;
result.add(new ExtLibInfo(lib, problem));
result.add(
new ExtLibInfo(libName, lib, libPath, libProg, new ArrayList<>(), problem));
} }
return result; return result;
} }
@@ -388,10 +305,6 @@ public class ExternalSymbolResolver implements Closeable {
* @throws CancelledException if cancelled * @throws CancelledException if cancelled
*/ */
private void resolveSymbolsToLibrary(ExtLibInfo extLib) throws CancelledException { private void resolveSymbolsToLibrary(ExtLibInfo extLib) throws CancelledException {
if (extLib.program == null) {
// can't do anything if the external library doesn't have a valid program associated
return;
}
ExternalManager externalManager = program.getExternalManager(); ExternalManager externalManager = program.getExternalManager();
SymbolTable symbolTable = program.getSymbolTable(); SymbolTable symbolTable = program.getSymbolTable();
@@ -409,7 +322,7 @@ public class ExternalSymbolResolver implements Closeable {
ExternalLocation extLoc = externalManager.getExternalLocation(s); ExternalLocation extLoc = externalManager.getExternalLocation(s);
String extLocName = String extLocName =
Objects.requireNonNullElse(extLoc.getOriginalImportedName(), extLoc.getLabel()); Objects.requireNonNullElse(extLoc.getOriginalImportedName(), extLoc.getLabel());
if (isExportedSymbol(extLib.program, extLocName)) { if (isExportedSymbol(program, extLocName)) {
try { try {
s.setNamespace(extLib.lib); s.setNamespace(extLib.lib);
idIterator.remove(); idIterator.remove();
@@ -443,6 +356,56 @@ public class ExternalSymbolResolver implements Closeable {
} }
return symbolIds; return symbolIds;
} }
private class ExtLibInfo {
final Library lib;
final List<String> resolvedSymbols = new ArrayList<>();
final Throwable problem;
/**
* Define external Library dependency associated with {@link ProgramSymbolResolver}
* instance.
* @param lib external library dependency
* @param problem exception which occured while accessing Library
*/
ExtLibInfo(Library lib, Throwable problem) {
if (program != lib.getSymbol().getProgram()) {
throw new AssertionError("Program mismatch");
}
this.lib = lib;
this.problem = problem;
}
String getName() {
return lib.getName();
}
String getProblemMessage() {
if (problem instanceof VersionException ve) {
return getVersionError(ve);
}
return problem != null ? problem.getMessage() : "";
}
String getVersionError(VersionException ve) {
String versionType = switch (ve.getVersionIndicator()) {
case VersionException.NEWER_VERSION -> " newer";
case VersionException.OLDER_VERSION -> "n older";
default -> "n unknown";
};
String upgradeMsg = ve.isUpgradable() ? " (upgrade is possible)" : "";
return "skipped: file was created with a%s version of Ghidra%s"
.formatted(versionType, upgradeMsg);
}
String getAssociatedProgramPath() {
return lib.getAssociatedProgramPath();
}
}
} }
/** /**
Ghidra/Features/Base/src/main/java/ghidra/program/util/SymbolMerge.java
+12 -3
View File
@@ -230,11 +230,20 @@ class SymbolMerge {
} }
else if (type == SymbolType.LIBRARY) { else if (type == SymbolType.LIBRARY) {
ExternalManager fromExtMgr = fromProgram.getExternalManager(); ExternalManager fromExtMgr = fromProgram.getExternalManager();
String path = fromExtMgr.getExternalLibraryPath(name); Library fromLib = fromExtMgr.getExternalLibrary(name);
String path = fromLib != null ? fromLib.getAssociatedProgramPath() : null;
int ordinal = fromLib != null ? fromExtMgr.getLibraryOrdinal(name) : -1;
ExternalManagerDB extMgr = (ExternalManagerDB) toProgram.getExternalManager(); ExternalManagerDB extMgr = (ExternalManagerDB) toProgram.getExternalManager();
extMgr.setExternalPath(name, path, source == SourceType.USER_DEFINED); Library newLib = extMgr.addExternalLibraryName(name, source);
symbol = toSymbolTable.getLibrarySymbol(name); symbol = newLib.getSymbol();
if (ordinal >= 0) {
extMgr.setLibraryOrdinal(name, ordinal);
}
if (path != null) {
extMgr.setExternalPath(name, path, source == SourceType.USER_DEFINED);
}
} }
else if (type == SymbolType.FUNCTION) { else if (type == SymbolType.FUNCTION) {
FunctionManager fromFunctionMgr = fromProgram.getFunctionManager(); FunctionManager fromFunctionMgr = fromProgram.getFunctionManager();
Ghidra/Features/Base/src/test.slow/java/ghidra/program/database/external/ExternalManagerDBTest.java
Vendored
+2 -3
View File
@@ -25,7 +25,6 @@ import ghidra.program.database.ProgramBuilder;
import ghidra.program.database.ProgramDB; import ghidra.program.database.ProgramDB;
import ghidra.program.model.address.Address; import ghidra.program.model.address.Address;
import ghidra.program.model.address.AddressSpace; import ghidra.program.model.address.AddressSpace;
import ghidra.program.model.listing.CircularDependencyException;
import ghidra.program.model.listing.Library; import ghidra.program.model.listing.Library;
import ghidra.program.model.symbol.*; import ghidra.program.model.symbol.*;
import ghidra.test.AbstractGhidraHeadedIntegrationTest; import ghidra.test.AbstractGhidraHeadedIntegrationTest;
@@ -47,7 +46,7 @@ public class ExternalManagerDBTest extends AbstractGhidraHeadedIntegrationTest {
public void setUp() throws Exception { public void setUp() throws Exception {
program = createDefaultProgram(testName.getMethodName(), ProgramBuilder._TOY, this); program = createDefaultProgram(testName.getMethodName(), ProgramBuilder._TOY, this);
space = program.getAddressFactory().getDefaultAddressSpace(); space = program.getAddressFactory().getDefaultAddressSpace();
extMgr = (ExternalManagerDB) program.getExternalManager(); extMgr = program.getExternalManager();
transactionID = program.startTransaction("Test"); transactionID = program.startTransaction("Test");
} }
@@ -336,7 +335,7 @@ public class ExternalManagerDBTest extends AbstractGhidraHeadedIntegrationTest {
@Test @Test
public void testOriginalImportName() public void testOriginalImportName()
throws InvalidInputException, DuplicateNameException, CircularDependencyException { throws InvalidInputException, DuplicateNameException {
ExternalLocation extLoc = ExternalLocation extLoc =
extMgr.addExtLocation("ext1", "foo", addr(1000), SourceType.IMPORTED); extMgr.addExtLocation("ext1", "foo", addr(1000), SourceType.IMPORTED);
Ghidra/Features/Sarif/src/main/java/sarif/managers/ExternalLibSarifMgr.java
+14 -20
View File
@@ -16,28 +16,15 @@
package sarif.managers; package sarif.managers;
import java.io.IOException; import java.io.IOException;
import java.util.ArrayList; import java.util.*;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import com.google.gson.JsonArray; import com.google.gson.JsonArray;
import ghidra.app.util.importer.MessageLog; import ghidra.app.util.importer.MessageLog;
import ghidra.program.model.address.Address; import ghidra.program.model.address.*;
import ghidra.program.model.address.AddressFormatException; import ghidra.program.model.listing.*;
import ghidra.program.model.address.AddressOverflowException; import ghidra.program.model.symbol.*;
import ghidra.program.model.listing.GhidraClass; import ghidra.util.exception.*;
import ghidra.program.model.listing.Library;
import ghidra.program.model.listing.Program;
import ghidra.program.model.symbol.ExternalLocation;
import ghidra.program.model.symbol.ExternalManager;
import ghidra.program.model.symbol.Namespace;
import ghidra.program.model.symbol.SourceType;
import ghidra.util.exception.CancelledException;
import ghidra.util.exception.DuplicateNameException;
import ghidra.util.exception.InvalidInputException;
import ghidra.util.task.TaskLauncher; import ghidra.util.task.TaskLauncher;
import ghidra.util.task.TaskMonitor; import ghidra.util.task.TaskMonitor;
import sarif.SarifProgramOptions; import sarif.SarifProgramOptions;
@@ -127,14 +114,21 @@ public class ExternalLibSarifMgr extends SarifMgr {
return; // already has a value--don't override it return; // already has a value--don't override it
} }
// NB: Can't use "DEFAULT" here or result.get("sourceType") which may be // NB: Can't use "DEFAULT" here or result.get("sourceType") which may be "DEFAULT"
// "DEFAULT"
String source = (String) result.get("sourceType"); String source = (String) result.get("sourceType");
SourceType sourceType = getSourceType(source); SourceType sourceType = getSourceType(source);
if (sourceType.equals(SourceType.DEFAULT)) { if (sourceType.equals(SourceType.DEFAULT)) {
sourceType = SourceType.IMPORTED; sourceType = SourceType.IMPORTED;
} }
library = extManager.addExternalLibraryName(progName, sourceType); library = extManager.addExternalLibraryName(progName, sourceType);
// NOTE: It is assumed that a full export/import will maintain the ordered sequence of Libraries
// NOTE: Skip setting library location within project which is unlikely to match-up properly
// String progPath = (String) result.get("location");
// if (progPath != null) {
// extManager.setExternalPath(progName, progPath, sourceType == SourceType.USER_DEFINED);
// }
} }
private void processExternalLocation(Map<String, Object> result) throws IOException { private void processExternalLocation(Map<String, Object> result) throws IOException {
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/ProgramDB.java
+3 -1
View File
@@ -120,8 +120,9 @@ public class ProgramDB extends DomainObjectAdapterDB implements Program, ChangeM
* 15-Sep-2025 - version 31 Code Mananger dropped Composites property map use * 15-Sep-2025 - version 31 Code Mananger dropped Composites property map use
* 19-Sep-2025 - version 32 Expanded number of SourceType values and record storage affecting * 19-Sep-2025 - version 32 Expanded number of SourceType values and record storage affecting
* SymbolDB, FunctionDB and RefListFlagsV0 * SymbolDB, FunctionDB and RefListFlagsV0
* 14-Apr-2026 - version 33 Introduced Library symbol ordinal assignment.
*/ */
static final int DB_VERSION = 32; static final int DB_VERSION = 33;
/** /**
* UPGRADE_REQUIRED_BFORE_VERSION should be changed to DB_VERSION anytime the * UPGRADE_REQUIRED_BFORE_VERSION should be changed to DB_VERSION anytime the
@@ -142,6 +143,7 @@ public class ProgramDB extends DomainObjectAdapterDB implements Program, ChangeM
public static final int COMPOUND_VARIABLE_STORAGE_ADDED_VERSION = 18; public static final int COMPOUND_VARIABLE_STORAGE_ADDED_VERSION = 18;
public static final int AUTO_PARAMETERS_ADDED_VERSION = 19; public static final int AUTO_PARAMETERS_ADDED_VERSION = 19;
public static final int RELOCATION_STATUS_ADDED_VERSION = 26; public static final int RELOCATION_STATUS_ADDED_VERSION = 26;
public static final int LIBRARY_ORDINAL_ASSIGNMENT_ADDED_VERSION = 33;
private static final String DATA_MAP_TABLE_NAME = "Program"; private static final String DATA_MAP_TABLE_NAME = "Program";
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/external/ExternalLocationDB.java
Vendored
+6
View File
@@ -55,6 +55,12 @@ public class ExternalLocationDB implements ExternalLocation {
return library != null ? library.getName() : "<UNKNOWN>"; return library != null ? library.getName() : "<UNKNOWN>";
} }
@Override
public String getExternalLibraryPath() {
Library library = getLibrary();
return library != null ? library.getAssociatedProgramPath() : null;
}
private Library getLibrary() { private Library getLibrary() {
Namespace parent = symbol.getParentNamespace(); Namespace parent = symbol.getParentNamespace();
while (parent != null && !(parent instanceof Library)) { while (parent != null && !(parent instanceof Library)) {
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/external/ExternalManagerDB.java
Vendored
+68 -67
View File
@@ -22,7 +22,6 @@ import org.apache.commons.lang3.StringUtils;
import db.*; import db.*;
import ghidra.framework.data.OpenMode; import ghidra.framework.data.OpenMode;
import ghidra.framework.store.FileSystem;
import ghidra.program.database.ManagerDB; import ghidra.program.database.ManagerDB;
import ghidra.program.database.ProgramDB; import ghidra.program.database.ProgramDB;
import ghidra.program.database.function.FunctionDB; import ghidra.program.database.function.FunctionDB;
@@ -126,7 +125,7 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
String name = rec.getString(OldExtNameAdapter.EXT_NAME_COL); String name = rec.getString(OldExtNameAdapter.EXT_NAME_COL);
try { try {
addExternalName(name, rec.getString(OldExtNameAdapter.EXT_PATHNAME_COL), doAddExternalName(name, rec.getString(OldExtNameAdapter.EXT_PATHNAME_COL),
SourceType.USER_DEFINED); SourceType.USER_DEFINED);
nameMap.put(rec.getKey(), name); nameMap.put(rec.getKey(), name);
} }
@@ -200,11 +199,8 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
SourceType sourceType) throws InvalidInputException, DuplicateNameException { SourceType sourceType) throws InvalidInputException, DuplicateNameException {
SourceType locSourceType = checkExternalLabel(extLabel, extAddr, sourceType); SourceType locSourceType = checkExternalLabel(extLabel, extAddr, sourceType);
try (Closeable c = lock.write()) { try (Closeable c = lock.write()) {
Library libraryScope = getLibraryScope(extLibraryName); Library library = addExternalLibraryName(extLibraryName, sourceType);
if (libraryScope == null) { return addExtLocation(library, extLabel, extAddr, false, locSourceType, true);
libraryScope = addExternalName(extLibraryName, null, sourceType);
}
return addExtLocation(libraryScope, extLabel, extAddr, false, locSourceType, true);
} }
} }
@@ -231,12 +227,9 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
SourceType sourceType) throws InvalidInputException, DuplicateNameException { SourceType sourceType) throws InvalidInputException, DuplicateNameException {
SourceType locSourceType = checkExternalLabel(extLabel, extAddr, sourceType); SourceType locSourceType = checkExternalLabel(extLabel, extAddr, sourceType);
try (Closeable c = lock.write()) { try (Closeable c = lock.write()) {
Library libraryScope = getLibraryScope(extLibraryName); Library library = addExternalLibraryName(extLibraryName,
if (libraryScope == null) {
libraryScope = addExternalName(extLibraryName, null,
sourceType != SourceType.DEFAULT ? sourceType : SourceType.ANALYSIS); sourceType != SourceType.DEFAULT ? sourceType : SourceType.ANALYSIS);
} return addExtLocation(library, extLabel, extAddr, true, locSourceType, true);
return addExtLocation(libraryScope, extLabel, extAddr, true, locSourceType, true);
} }
} }
@@ -272,16 +265,11 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
Address extAddr, boolean isFunction, SourceType sourceType, boolean reuseExisting) Address extAddr, boolean isFunction, SourceType sourceType, boolean reuseExisting)
throws InvalidInputException { throws InvalidInputException {
if (extNamespace == null) { if (extNamespace == null) {
extNamespace = getLibraryScope(Library.UNKNOWN); try {
if (extNamespace == null) { extNamespace = addExternalLibraryName(Library.UNKNOWN, SourceType.ANALYSIS);
try { }
extNamespace = addExternalLibraryName(Library.UNKNOWN, SourceType.ANALYSIS); catch (InvalidInputException | DuplicateNameException e) {
} throw new AssertionError(e); // reserved name should be OK
catch (DuplicateNameException e) {
// TODO: really need to reserve the unknown namespace name
throw new InvalidInputException(
"Failed to establish " + Library.UNKNOWN + " library");
}
} }
} }
else if (!extNamespace.isExternal()) { else if (!extNamespace.isExternal()) {
@@ -471,7 +459,7 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
@Override @Override
public Set<ExternalLocation> getExternalLocations(String libraryName, String label) { public Set<ExternalLocation> getExternalLocations(String libraryName, String label) {
Library library = getLibraryScope(libraryName); Library library = getExternalLibrary(libraryName);
if (library == null && !StringUtils.isBlank(libraryName)) { if (library == null && !StringUtils.isBlank(libraryName)) {
return Set.of(); return Set.of();
} }
@@ -489,7 +477,7 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
@Override @Override
public ExternalLocation getUniqueExternalLocation(String libraryName, String label) { public ExternalLocation getUniqueExternalLocation(String libraryName, String label) {
Library library = getLibraryScope(libraryName); Library library = getExternalLibrary(libraryName);
if (library == null && !StringUtils.isBlank(libraryName)) { if (library == null && !StringUtils.isBlank(libraryName)) {
return null; return null;
} }
@@ -598,12 +586,16 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
} }
@Override @Override
public void updateExternalLibraryName(String oldName, String newName, SourceType source) public boolean updateExternalLibraryName(String oldName, String newName, SourceType source)
throws DuplicateNameException, InvalidInputException { throws DuplicateNameException, InvalidInputException {
Symbol s = symbolMgr.getLibrarySymbol(oldName); try (Closeable c = lock.write()) {
if (s != null) { Symbol s = symbolMgr.getLibrarySymbol(oldName);
s.setName(newName, source); if (s != null) {
s.setName(newName, source);
return true;
}
} }
return false;
} }
@Override @Override
@@ -614,34 +606,37 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
if (librarySymbol != null) { if (librarySymbol != null) {
return (Library) librarySymbol.getObject(); return (Library) librarySymbol.getObject();
} }
return addExternalName(name, null, source); return doAddExternalName(name, null, source);
} }
} }
private Library addExternalName(String name, String pathname, SourceType source) private Library doAddExternalName(String name, String pathname, SourceType source)
throws DuplicateNameException, InvalidInputException { throws DuplicateNameException, InvalidInputException {
SymbolDB s = symbolMgr.createLibrarySymbol(name, pathname, source); SymbolDB s = symbolMgr.createLibrarySymbol(name, pathname, source);
return (Library) s.getObject(); return (Library) s.getObject();
} }
private Library getLibraryScope(String name) {
LibrarySymbol s = symbolMgr.getLibrarySymbol(name);
return s == null ? null : s.getObject();
}
@Override @Override
public boolean contains(String libraryName) { public boolean contains(String libraryName) {
return symbolMgr.getLibrarySymbol(libraryName) != null; return symbolMgr.getLibrarySymbol(libraryName) != null;
} }
@Override
public List<Library> getLibraries() {
try (Closeable c = lock.read()) {
List<Library> orderedLibraries = new ArrayList<>();
for (LibrarySymbol libSym : symbolMgr.getLibrarySymbolList()) {
orderedLibraries.add(libSym.getObject());
}
return orderedLibraries;
}
}
@Override @Override
public String[] getExternalLibraryNames() { public String[] getExternalLibraryNames() {
ArrayList<String> list = new ArrayList<>(); ArrayList<String> list = new ArrayList<>();
Symbol[] syms = symbolMgr.getSymbols(Address.NO_ADDRESS); for (LibrarySymbol libSym : symbolMgr.getLibrarySymbolList()) {
for (Symbol s : syms) { list.add(libSym.getName());
if (s.getSymbolType() == SymbolType.LIBRARY) {
list.add(s.getName());
}
} }
String[] names = new String[list.size()]; String[] names = new String[list.size()];
list.toArray(names); list.toArray(names);
@@ -650,8 +645,10 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
@Override @Override
public Library getExternalLibrary(String name) { public Library getExternalLibrary(String name) {
Symbol s = symbolMgr.getLibrarySymbol(name); try (Closeable c = lock.read()) {
return s != null ? (Library) s.getObject() : null; Symbol s = symbolMgr.getLibrarySymbol(name);
return s != null ? (Library) s.getObject() : null;
}
} }
@Override @Override
@@ -672,34 +669,38 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
return; return;
} }
validateExternalPath(externalPath); LibrarySymbol.validateExternalPath(externalPath);
try (Closeable c = lock.write()) { try (Closeable c = lock.write()) {
LibrarySymbol s = symbolMgr.getLibrarySymbol(externalName); Library library = addExternalLibraryName(externalName,
if (s == null) { userDefined ? SourceType.USER_DEFINED : SourceType.IMPORTED);
try { LibrarySymbol libSym = (LibrarySymbol) library.getSymbol();
addExternalName(externalName, externalPath, libSym.setExternalLibraryPath(externalPath);
userDefined ? SourceType.USER_DEFINED : SourceType.IMPORTED); }
} catch (DuplicateNameException e) {
catch (DuplicateNameException e) { // ignore - new externalName conflicts with another namespace
throw new AssertException(e);
}
}
else {
s.setExternalLibraryPath(externalPath);
}
} }
} }
private void validateExternalPath(String path) throws InvalidInputException { @Override
if (path == null) { public int getLibraryOrdinal(String libraryName) {
return; // null is an allowed value (used to clear) LibrarySymbol libSym = symbolMgr.getLibrarySymbol(libraryName);
} return libSym != null ? libSym.getOrdinal() : -1;
}
int len = path.length(); @Override
if (len == 0 || path.charAt(0) != FileSystem.SEPARATOR_CHAR) { public int setLibraryOrdinal(String libraryName, int ordinal) {
throw new InvalidInputException( if (Library.UNKNOWN.equals(libraryName)) {
"Absolute path must begin with '" + FileSystem.SEPARATOR_CHAR + "'"); Msg.warn(this, "Ignoring external library ordinal assignment for " + libraryName);
return -1;
}
try (Closeable c = lock.write()) {
LibrarySymbol libSym = symbolMgr.getLibrarySymbol(libraryName);
if (libSym == null) {
return -1;
}
libSym.setOrdinal(ordinal);
return libSym.getOrdinal();
} }
} }
@@ -740,9 +741,9 @@ public class ExternalManagerDB implements ManagerDB, ExternalManager {
@Override @Override
public ExternalLocationIterator getExternalLocations(String externalName) { public ExternalLocationIterator getExternalLocations(String externalName) {
Library scope = getLibraryScope(externalName); Library library = getExternalLibrary(externalName);
if (scope != null) { if (library != null) {
return new ExternalLocationDBIterator(symbolMgr.getSymbols(scope)); return new ExternalLocationDBIterator(symbolMgr.getSymbols(library));
} }
return new ExternalLocationDBIterator(); return new ExternalLocationDBIterator();
} }
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/LibraryDB.java
+5
View File
@@ -99,6 +99,11 @@ class LibraryDB implements Library {
return symbol.getExternalLibraryPath(); return symbol.getExternalLibraryPath();
} }
@Override
public void setAssociatedProgramPath(String programPath) throws InvalidInputException {
symbol.setExternalLibraryPath(programPath);
}
@Override @Override
public boolean isExternal() { public boolean isExternal() {
return true; return true;
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/LibrarySymbol.java
+183 -24
View File
@@ -15,7 +15,11 @@
*/ */
package ghidra.program.database.symbol; package ghidra.program.database.symbol;
import java.util.*;
import db.DBRecord; import db.DBRecord;
import db.Field;
import ghidra.framework.store.FileSystem;
import ghidra.program.model.address.Address; import ghidra.program.model.address.Address;
import ghidra.program.model.listing.CircularDependencyException; import ghidra.program.model.listing.CircularDependencyException;
import ghidra.program.model.listing.Library; import ghidra.program.model.listing.Library;
@@ -32,7 +36,7 @@ import ghidra.util.exception.InvalidInputException;
* Symbol data usage: * Symbol data usage:
* String stringData - associated program project file path * String stringData - associated program project file path
*/ */
public class LibrarySymbol extends SymbolDB { public class LibrarySymbol extends SymbolDB implements Comparable<LibrarySymbol> {
private LibraryDB library; private LibraryDB library;
@@ -46,35 +50,69 @@ public class LibrarySymbol extends SymbolDB {
} }
@Override @Override
public void setName(String newName, SourceType source) public void setNameAndNamespace(String newName, Namespace newNamespace, SourceType source)
throws DuplicateNameException, InvalidInputException { throws DuplicateNameException, InvalidInputException, CircularDependencyException {
String oldName = getName(); String oldName = getName();
if (Library.UNKNOWN.equals(oldName)) { if (Library.UNKNOWN.equals(oldName)) {
Msg.warn(this, "Unable to change name of " + Library.UNKNOWN + " Library"); Msg.warn(this, "Unable to change name of " + Library.UNKNOWN + " Library");
return; return;
} }
if (newNamespace.getID() != Namespace.GLOBAL_NAMESPACE_ID) {
throw new InvalidInputException("Namespace \"" + newNamespace.getName(true) +
"\" is not valid for library " + getName());
}
super.setName(newName, source); try (Closeable c = lock.write()) {
if (!oldName.equals(getName())) { super.setNameAndNamespace(newName, newNamespace, source);
symbolMgr.getProgram()
.setObjChanged(ProgramEvent.EXTERNAL_NAME_CHANGED, (Address) null, null, if (!oldName.equals(getName())) {
oldName, newName); symbolMgr.getProgram()
.setObjChanged(ProgramEvent.EXTERNAL_NAME_CHANGED, (Address) null, this,
oldName, newName);
}
if (Library.UNKNOWN.equals(newName)) {
symbolMgr.adjustLibraryOrdinals(this, 0);
setExternalLibraryPath(null); // clear file path for UNKNOWN lib
}
} }
} }
@Override @Override
public void setNameAndNamespace(String newName, Namespace newNamespace, SourceType source) public boolean delete() {
throws DuplicateNameException, InvalidInputException, CircularDependencyException { try (Closeable c = lock.write()) {
String oldName = getName();
super.setNameAndNamespace(newName, newNamespace, source); // Pre-fetch library symbol list to facilitate ordinal reassignments after removal
int ordinal = getOrdinal();
List<LibrarySymbol> libSymList = new ArrayList<>(symbolMgr.getLibrarySymbolList());
if (!oldName.equals(getName())) { if (super.delete()) {
symbolMgr.getProgram() // Perform ordinal reassignments for remaining library symbols if needed.
.setObjChanged(ProgramEvent.EXTERNAL_NAME_CHANGED, (Address) null, null, // It is expected that all ordinals are accounted for in cached library list
oldName, newName); // but to be safe we will perform brute-force search if mismatch and ignore
// if not found in list;
LibrarySymbol s = libSymList.get(ordinal);
if (s != this) {
String libName = record.getString(SymbolDatabaseAdapter.SYMBOL_NAME_COL);
Msg.error(this,
"Library symbol list did not contain removed symbol: " + libName);
for (ordinal = 0; ordinal < libSymList.size(); ordinal++) {
s = libSymList.get(ordinal);
if (s == this) {
break;
}
}
}
if (s == this) {
libSymList.remove(ordinal);
symbolMgr.assignLibraryOrdinals(libSymList, false);
}
return true;
}
} }
return false;
} }
@Override @Override
@@ -111,6 +149,76 @@ public class LibrarySymbol extends SymbolDB {
SymbolType.LIBRARY.isValidParent(symbolMgr.getProgram(), parent, address, isExternal()); SymbolType.LIBRARY.isValidParent(symbolMgr.getProgram(), parent, address, isExternal());
} }
/**
* {@return Library's ordinal placement within ordered library list.}
*/
public int getOrdinal() {
// NOTE: This method must not be used by compareTo since it may invoke it
validate(lock);
int ordinal = doGetOrdinalFromRecord();
if (ordinal < 0) {
// NOTE: this method invocation relies on the use of the compareTo method
ordinal = symbolMgr.computeLibraryOrdinal(this);
}
return ordinal;
}
/**
* Get this Library's ordinal as stored in the database. A value of -1 is returned if one has
* not yet been established in which case the symbol ID should be used for sort while ensuring
* {@link Library#UNKNOWN} always sorts as first.
*
* @return Library symbol stored ordinal or -1 if not yet stored.
*/
int doGetOrdinalFromRecord() {
Field fieldValue = record.getFieldValue(SymbolDatabaseAdapter.SYMBOL_LIB_ORDINAL_COL);
return fieldValue != null ? fieldValue.getIntValue() : -1;
}
/**
* Set the Library ordinal.
* Other Library ordinals will be adjusted if displaced by ordinal change.
* No change is made if this Library symbol corresponds to {@link Library#UNKNOWN} Library.
*
* @param ordinal positive greater or equal to 0.
* @throws IllegalArgumentException if a negative ordinal is specified
*/
public void setOrdinal(int ordinal) {
if (ordinal < 0) {
throw new IllegalArgumentException("Non-negative ordinal is required");
}
if (Library.UNKNOWN.equals(getName())) {
return; // ordinal change ignored for UNKNOWN Library
}
try (Closeable c = lock.write()) {
checkDeleted();
if (ordinal == 0) {
// Cannot displace UNKNOWN Library which may reside at ordinal 0
LibrarySymbol displacedLibSym = symbolMgr.getLibrarySymbolByOrdinal(0);
if (displacedLibSym != null && Library.UNKNOWN.equals(displacedLibSym.getName())) {
return;
}
}
symbolMgr.adjustLibraryOrdinals(this, ordinal);
}
}
void doSetOrdinal(int newOrdinal, boolean notify) {
if (newOrdinal < 0) {
throw new IllegalArgumentException("Unsupported ordinal assignment: " + newOrdinal);
}
int oldOrdinal = doGetOrdinalFromRecord();
if (oldOrdinal == newOrdinal) {
return;
}
record.setIntValue(SymbolDatabaseAdapter.SYMBOL_LIB_ORDINAL_COL, newOrdinal);
updateRecord();
if (notify) {
symbolMgr.symbolDataChanged(this);
}
}
/** /**
* {@return the library program path within the project (may be null)} * {@return the library program path within the project (may be null)}
*/ */
@@ -121,22 +229,73 @@ public class LibrarySymbol extends SymbolDB {
/** /**
* Set the library program path within the project. * Set the library program path within the project.
* The {@link Library#UNKNOWN} Library path may only be cleared.
*
* @param libraryPath library program path or null to clear * @param libraryPath library program path or null to clear
* @throws InvalidInputException if an invalid project file path is specified
*/ */
public void setExternalLibraryPath(String libraryPath) { public void setExternalLibraryPath(String libraryPath) throws InvalidInputException {
String oldPath = getExternalLibraryPath();
try (Closeable c = lock.write()) { try (Closeable c = lock.write()) {
checkDeleted(); checkDeleted();
setRecordFields(record, libraryPath); if (Library.UNKNOWN.equals(getName())) {
libraryPath = null;
}
validateExternalPath(libraryPath);
String oldPath = record.getString(SymbolDatabaseAdapter.SYMBOL_LIBPATH_COL);
if (Objects.equals(oldPath, libraryPath)) {
return;
}
record.setString(SymbolDatabaseAdapter.SYMBOL_LIBPATH_COL, libraryPath);
updateRecord(); updateRecord();
} }
symbolMgr.getProgram() symbolMgr.symbolDataChanged(this);
.setObjChanged(ProgramEvent.EXTERNAL_PATH_CHANGED, getName(), oldPath, libraryPath);
} }
static void setRecordFields(DBRecord record, String libraryPath) { /**
* Perform path validation for an external library path within the project
* @param path external library path within the project (null is allowed for clearing path)
* @throws InvalidInputException if path is invalid
*/
public static void validateExternalPath(String path) throws InvalidInputException {
if (path == null) {
return; // null is an allowed value (used to clear)
}
int len = path.length();
if (len == 0 || path.charAt(0) != FileSystem.SEPARATOR_CHAR) {
throw new InvalidInputException(
"Absolute path must begin with '" + FileSystem.SEPARATOR_CHAR + "'");
}
}
/**
* Set library symbol record fields during symbol creation
* @param record new symbol record
* @param ordinal library symbol ordinal
* @param libraryPath library path or null
*/
static void setRecordFields(DBRecord record, int ordinal, String libraryPath) {
// NOTE: method use must be limited since ordinal re-assignments of affected Libraries
// is not handled here.
record.setIntValue(SymbolDatabaseAdapter.SYMBOL_LIB_ORDINAL_COL, ordinal);
record.setString(SymbolDatabaseAdapter.SYMBOL_LIBPATH_COL, libraryPath); record.setString(SymbolDatabaseAdapter.SYMBOL_LIBPATH_COL, libraryPath);
} }
@Override
public int compareTo(LibrarySymbol o) {
validate(lock);
o.validate(lock);
// NOTE: this method is not intended to be used between symbols from different programs
// where one may have stored ordinals and the other may not in which case this comparison
// would be invalid. For a single program it is required that all library symbols either
// have an assigned ordinal or do not in which cases symbol ID comparison is used
int c = Long.compare(doGetOrdinalFromRecord(), o.doGetOrdinalFromRecord());
if (c == 0) {
// Handles case where all library symbols report a -1 ordinal (not yet assigned)
// UNKNOWN Library placement is arbitrary in this case.
c = Long.compare(getID(), o.getID());
}
return c;
}
} }
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapter.java
+15 -6
View File
@@ -16,7 +16,8 @@
package ghidra.program.database.symbol; package ghidra.program.database.symbol;
import java.io.IOException; import java.io.IOException;
import java.util.*; import java.util.Objects;
import java.util.Set;
import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.StringUtils;
@@ -52,8 +53,9 @@ abstract class SymbolDatabaseAdapter {
static final int SYMBOL_EXTERNAL_PROG_ADDR_COL = 10; static final int SYMBOL_EXTERNAL_PROG_ADDR_COL = 10;
static final int SYMBOL_COMMENT_COL = 11; static final int SYMBOL_COMMENT_COL = 11;
static final int SYMBOL_LIBPATH_COL = 12; static final int SYMBOL_LIBPATH_COL = 12;
static final int SYMBOL_LIB_ORDINAL_COL = 13;
static final Schema SYMBOL_SCHEMA = SymbolDatabaseAdapterV4.V4_SYMBOL_SCHEMA; static final Schema SYMBOL_SCHEMA = SymbolDatabaseAdapterV5.V5_SYMBOL_SCHEMA;
// Bits 0, 1 and 3 are used for the source of the symbol. // Bits 0, 1 and 3 are used for the source of the symbol.
// NOTE: On the next V5 adapter revision the source type bits should be made contiguous // NOTE: On the next V5 adapter revision the source type bits should be made contiguous
@@ -96,11 +98,11 @@ abstract class SymbolDatabaseAdapter {
throws VersionException, CancelledException, IOException { throws VersionException, CancelledException, IOException {
if (openMode == OpenMode.CREATE) { if (openMode == OpenMode.CREATE) {
return new SymbolDatabaseAdapterV4(dbHandle, addrMap, true); return new SymbolDatabaseAdapterV5(dbHandle, addrMap, true);
} }
try { try {
SymbolDatabaseAdapter adapter = new SymbolDatabaseAdapterV4(dbHandle, addrMap, false); SymbolDatabaseAdapter adapter = new SymbolDatabaseAdapterV5(dbHandle, addrMap, false);
return adapter; return adapter;
} }
catch (VersionException e) { catch (VersionException e) {
@@ -122,6 +124,13 @@ abstract class SymbolDatabaseAdapter {
private static SymbolDatabaseAdapter findReadOnlyAdapter(DBHandle handle, AddressMap addrMap) private static SymbolDatabaseAdapter findReadOnlyAdapter(DBHandle handle, AddressMap addrMap)
throws VersionException { throws VersionException {
try {
return new SymbolDatabaseAdapterV4(handle, addrMap.getOldAddressMap());
}
catch (VersionException e) {
// failed try older version
}
try { try {
return new SymbolDatabaseAdapterV3(handle, addrMap.getOldAddressMap()); return new SymbolDatabaseAdapterV3(handle, addrMap.getOldAddressMap());
} }
@@ -168,7 +177,7 @@ abstract class SymbolDatabaseAdapter {
dbHandle.deleteTable(SYMBOL_TABLE_NAME); dbHandle.deleteTable(SYMBOL_TABLE_NAME);
SymbolDatabaseAdapter newAdapter = new SymbolDatabaseAdapterV4(dbHandle, addrMap, true); SymbolDatabaseAdapter newAdapter = new SymbolDatabaseAdapterV5(dbHandle, addrMap, true);
copyTempToNewAdapter(tmpAdapter, newAdapter, monitor); copyTempToNewAdapter(tmpAdapter, newAdapter, monitor);
return newAdapter; return newAdapter;
@@ -194,7 +203,7 @@ abstract class SymbolDatabaseAdapter {
((SymbolDatabaseAdapterV0) oldAdapter).extractLocalSymbols(tmpHandle, monitor); ((SymbolDatabaseAdapterV0) oldAdapter).extractLocalSymbols(tmpHandle, monitor);
} }
SymbolDatabaseAdapterV4 tmpAdapter = new SymbolDatabaseAdapterV4(tmpHandle, addrMap, true); SymbolDatabaseAdapterV5 tmpAdapter = new SymbolDatabaseAdapterV5(tmpHandle, addrMap, true);
RecordIterator iter = oldAdapter.getSymbols(); RecordIterator iter = oldAdapter.getSymbols();
while (iter.hasNext()) { while (iter.hasNext()) {
monitor.checkCancelled(); monitor.checkCancelled();
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapterV2.java
+11 -11
View File
@@ -97,7 +97,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) { if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return Field.EMPTY_ARRAY; return Field.EMPTY_ARRAY;
} }
return symbolTable.findRecords(new LongField(key), SYMBOL_ADDR_COL); return symbolTable.findRecords(new LongField(key), V2_SYMBOL_ADDR_COL);
} }
@Override @Override
@@ -113,7 +113,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
@Override @Override
RecordIterator getSymbolsByAddress(boolean forward) throws IOException { RecordIterator getSymbolsByAddress(boolean forward) throws IOException {
KeyToRecordIterator it = new KeyToRecordIterator(symbolTable, KeyToRecordIterator it = new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, SYMBOL_ADDR_COL, addrMap, forward)); new AddressIndexPrimaryKeyIterator(symbolTable, V2_SYMBOL_ADDR_COL, addrMap, forward));
return new V2ConvertedRecordIterator(it); return new V2ConvertedRecordIterator(it);
} }
@@ -121,7 +121,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException { RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException {
KeyToRecordIterator it = KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, startAddr, forward)); V2_SYMBOL_ADDR_COL, addrMap, startAddr, forward));
return new V2ConvertedRecordIterator(it); return new V2ConvertedRecordIterator(it);
} }
@@ -139,7 +139,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException { RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException {
KeyToRecordIterator it = KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, start, end, forward)); V2_SYMBOL_ADDR_COL, addrMap, start, end, forward));
return new V2ConvertedRecordIterator(it); return new V2ConvertedRecordIterator(it);
} }
@@ -147,7 +147,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException { RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException {
KeyToRecordIterator it = KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward)); V2_SYMBOL_ADDR_COL, addrMap, set, forward));
return new V2ConvertedRecordIterator(it); return new V2ConvertedRecordIterator(it);
} }
@@ -155,7 +155,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
RecordIterator getPrimarySymbols(AddressSetView set, boolean forward) throws IOException { RecordIterator getPrimarySymbols(AddressSetView set, boolean forward) throws IOException {
KeyToRecordIterator it = KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward)); V2_SYMBOL_ADDR_COL, addrMap, set, forward));
return getPrimaryFilterRecordIterator(new V2ConvertedRecordIterator(it)); return getPrimaryFilterRecordIterator(new V2ConvertedRecordIterator(it));
} }
@@ -222,21 +222,21 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
@Override @Override
RecordIterator getSymbolsByNamespace(long id) throws IOException { RecordIterator getSymbolsByNamespace(long id) throws IOException {
LongField field = new LongField(id); LongField field = new LongField(id);
RecordIterator it = symbolTable.indexIterator(SYMBOL_PARENT_ID_COL, field, field, true); RecordIterator it = symbolTable.indexIterator(V2_SYMBOL_PARENT_ID_COL, field, field, true);
return new V2ConvertedRecordIterator(it); return new V2ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator getSymbolsByName(String name) throws IOException { RecordIterator getSymbolsByName(String name) throws IOException {
StringField field = new StringField(name); StringField field = new StringField(name);
RecordIterator it = symbolTable.indexIterator(SYMBOL_NAME_COL, field, field, true); RecordIterator it = symbolTable.indexIterator(V2_SYMBOL_NAME_COL, field, field, true);
return new V2ConvertedRecordIterator(it); return new V2ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator scanSymbolsByName(String startName) throws IOException { RecordIterator scanSymbolsByName(String startName) throws IOException {
StringField field = new StringField(startName); StringField field = new StringField(startName);
RecordIterator it = symbolTable.indexIterator(SYMBOL_NAME_COL, field, null, true); RecordIterator it = symbolTable.indexIterator(V2_SYMBOL_NAME_COL, field, null, true);
return new V2ConvertedRecordIterator(it); return new V2ConvertedRecordIterator(it);
} }
@@ -262,7 +262,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
Address getMaxSymbolAddress(AddressSpace space) throws IOException { Address getMaxSymbolAddress(AddressSpace space) throws IOException {
if (space.isMemorySpace()) { if (space.isMemorySpace()) {
AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable, AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false); V2_SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
if (addressKeyIterator.hasNext()) { if (addressKeyIterator.hasNext()) {
return addrMap.decodeAddress(addressKeyIterator.next()); return addrMap.decodeAddress(addressKeyIterator.next());
} }
@@ -270,7 +270,7 @@ class SymbolDatabaseAdapterV2 extends SymbolDatabaseAdapter {
else { else {
LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false)); LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false));
DBFieldIterator iterator = DBFieldIterator iterator =
symbolTable.indexFieldIterator(null, max, false, SYMBOL_ADDR_COL); symbolTable.indexFieldIterator(null, max, false, V2_SYMBOL_ADDR_COL);
if (iterator.hasPrevious()) { if (iterator.hasPrevious()) {
LongField val = (LongField) iterator.previous(); LongField val = (LongField) iterator.previous();
Address addr = addrMap.decodeAddress(val.getLongValue()); Address addr = addrMap.decodeAddress(val.getLongValue());
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapterV3.java
+17 -17
View File
@@ -110,7 +110,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) { if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return false; return false;
} }
return symbolTable.hasRecord(new LongField(key), SYMBOL_ADDR_COL); return symbolTable.hasRecord(new LongField(key), V3_SYMBOL_ADDR_COL);
} }
@Override @Override
@@ -119,7 +119,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) { if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return Field.EMPTY_ARRAY; return Field.EMPTY_ARRAY;
} }
return symbolTable.findRecords(new LongField(key), SYMBOL_ADDR_COL); return symbolTable.findRecords(new LongField(key), V3_SYMBOL_ADDR_COL);
} }
@Override @Override
@@ -135,7 +135,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
@Override @Override
RecordIterator getSymbolsByAddress(boolean forward) throws IOException { RecordIterator getSymbolsByAddress(boolean forward) throws IOException {
KeyToRecordIterator it = new KeyToRecordIterator(symbolTable, KeyToRecordIterator it = new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, SYMBOL_ADDR_COL, addrMap, forward)); new AddressIndexPrimaryKeyIterator(symbolTable, V3_SYMBOL_ADDR_COL, addrMap, forward));
return new V3ConvertedRecordIterator(it); return new V3ConvertedRecordIterator(it);
} }
@@ -143,7 +143,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException { RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException {
KeyToRecordIterator it = KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, startAddr, forward)); V3_SYMBOL_ADDR_COL, addrMap, startAddr, forward));
return new V3ConvertedRecordIterator(it); return new V3ConvertedRecordIterator(it);
} }
@@ -161,7 +161,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException { RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException {
KeyToRecordIterator it = KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, start, end, forward)); V3_SYMBOL_ADDR_COL, addrMap, start, end, forward));
return new V3ConvertedRecordIterator(it); return new V3ConvertedRecordIterator(it);
} }
@@ -169,7 +169,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException { RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException {
KeyToRecordIterator it = KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward)); V3_SYMBOL_ADDR_COL, addrMap, set, forward));
return new V3ConvertedRecordIterator(it); return new V3ConvertedRecordIterator(it);
} }
@@ -178,14 +178,14 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
throws IOException { throws IOException {
KeyToRecordIterator it = KeyToRecordIterator it =
new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, set, forward)); V3_SYMBOL_PRIMARY_COL, addrMap, set, forward));
return new V3ConvertedRecordIterator(it); return new V3ConvertedRecordIterator(it);
} }
@Override @Override
protected DBRecord getPrimarySymbol(Address address) throws IOException { protected DBRecord getPrimarySymbol(Address address) throws IOException {
AddressIndexPrimaryKeyIterator it = new AddressIndexPrimaryKeyIterator(symbolTable, AddressIndexPrimaryKeyIterator it = new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, address, address, true); V3_SYMBOL_PRIMARY_COL, addrMap, address, address, true);
if (it.hasNext()) { if (it.hasNext()) {
return convertV3Record(symbolTable.getRecord(it.next())); return convertV3Record(symbolTable.getRecord(it.next()));
} }
@@ -257,21 +257,21 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
@Override @Override
RecordIterator getSymbolsByNamespace(long id) throws IOException { RecordIterator getSymbolsByNamespace(long id) throws IOException {
LongField field = new LongField(id); LongField field = new LongField(id);
RecordIterator it = symbolTable.indexIterator(SYMBOL_PARENT_ID_COL, field, field, true); RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_PARENT_ID_COL, field, field, true);
return new V3ConvertedRecordIterator(it); return new V3ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator getSymbolsByName(String name) throws IOException { RecordIterator getSymbolsByName(String name) throws IOException {
StringField field = new StringField(name); StringField field = new StringField(name);
RecordIterator it = symbolTable.indexIterator(SYMBOL_NAME_COL, field, field, true); RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_NAME_COL, field, field, true);
return new V3ConvertedRecordIterator(it); return new V3ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator scanSymbolsByName(String startName) throws IOException { RecordIterator scanSymbolsByName(String startName) throws IOException {
StringField field = new StringField(startName); StringField field = new StringField(startName);
RecordIterator it = symbolTable.indexIterator(SYMBOL_NAME_COL, field, null, true); RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_NAME_COL, field, null, true);
return new V3ConvertedRecordIterator(it); return new V3ConvertedRecordIterator(it);
} }
@@ -286,7 +286,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
Field end = computeLocatorHash(name, id, MAX_ADDRESS_OFFSET); Field end = computeLocatorHash(name, id, MAX_ADDRESS_OFFSET);
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, start, end, true); RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_HASH_COL, start, end, true);
it = new V3ConvertedRecordIterator(it); it = new V3ConvertedRecordIterator(it);
RecordIterator filtered = getNameAndNamespaceFilterIterator(name, id, it); RecordIterator filtered = getNameAndNamespaceFilterIterator(name, id, it);
@@ -300,7 +300,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
if (search == null) { if (search == null) {
return null; return null;
} }
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, search, search, true); RecordIterator it = symbolTable.indexIterator(V3_SYMBOL_HASH_COL, search, search, true);
it = new V3ConvertedRecordIterator(it); it = new V3ConvertedRecordIterator(it);
RecordIterator filtered = RecordIterator filtered =
@@ -315,7 +315,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
Address getMaxSymbolAddress(AddressSpace space) throws IOException { Address getMaxSymbolAddress(AddressSpace space) throws IOException {
if (space.isMemorySpace()) { if (space.isMemorySpace()) {
AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable, AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false); V3_SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
if (addressKeyIterator.hasNext()) { if (addressKeyIterator.hasNext()) {
return addrMap.decodeAddress(addressKeyIterator.next()); return addrMap.decodeAddress(addressKeyIterator.next());
} }
@@ -323,7 +323,7 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
else { else {
LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false)); LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false));
DBFieldIterator iterator = DBFieldIterator iterator =
symbolTable.indexFieldIterator(null, max, false, SYMBOL_ADDR_COL); symbolTable.indexFieldIterator(null, max, false, V3_SYMBOL_ADDR_COL);
if (iterator.hasPrevious()) { if (iterator.hasPrevious()) {
LongField val = (LongField) iterator.previous(); LongField val = (LongField) iterator.previous();
Address addr = addrMap.decodeAddress(val.getLongValue()); Address addr = addrMap.decodeAddress(val.getLongValue());
@@ -341,8 +341,8 @@ class SymbolDatabaseAdapterV3 extends SymbolDatabaseAdapter {
} }
/** /**
* Returns a record matching the current database schema from the version 2 record. * Returns a record matching the current database schema from the version 3 record.
* @param record the record matching the version 2 schema. * @param record the record matching the version 3 schema.
* @return a current symbol record. * @return a current symbol record.
*/ */
private DBRecord convertV3Record(DBRecord record) { private DBRecord convertV3Record(DBRecord record) {
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapterV4.java
+164 -136
View File
@@ -16,13 +16,11 @@
package ghidra.program.database.symbol; package ghidra.program.database.symbol;
import java.io.IOException; import java.io.IOException;
import java.util.HashSet;
import java.util.Set; import java.util.Set;
import db.*; import db.*;
import ghidra.program.database.map.*; import ghidra.program.database.map.*;
import ghidra.program.database.util.EmptyRecordIterator; import ghidra.program.database.util.EmptyRecordIterator;
import ghidra.program.database.util.RecordFilter;
import ghidra.program.model.address.*; import ghidra.program.model.address.*;
import ghidra.program.model.symbol.SourceType; import ghidra.program.model.symbol.SourceType;
import ghidra.program.model.symbol.SymbolType; import ghidra.program.model.symbol.SymbolType;
@@ -31,7 +29,7 @@ import ghidra.util.exception.VersionException;
import ghidra.util.task.TaskMonitor; import ghidra.util.task.TaskMonitor;
/** /**
* SymbolDatabaseAdapter for version 3 * SymbolDatabaseAdapter for version 4
* *
* This version added additional sparse columns to store optional data specific to certain * This version added additional sparse columns to store optional data specific to certain
* symbol types. The adhoc string data column was eliminated. * symbol types. The adhoc string data column was eliminated.
@@ -48,94 +46,64 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
// allows us to index this field and quickly find the primary symbols. The field is sparse // allows us to index this field and quickly find the primary symbols. The field is sparse
// so that non-primary symbols don't consume any space for this field. // so that non-primary symbols don't consume any space for this field.
static final Schema V4_SYMBOL_SCHEMA = new Schema(SYMBOL_VERSION, "Key", /* Do not remove the following commented out schema! It shows the version 4 symbol table schema. */
new Field[] { StringField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE, // static final Schema V4_SYMBOL_SCHEMA = new Schema(SYMBOL_VERSION, "Key",
ByteField.INSTANCE, ByteField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE, // new Field[] { StringField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
LongField.INSTANCE, IntField.INSTANCE, StringField.INSTANCE, StringField.INSTANCE, // ByteField.INSTANCE, ByteField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
StringField.INSTANCE, StringField.INSTANCE }, // LongField.INSTANCE, IntField.INSTANCE, StringField.INSTANCE, StringField.INSTANCE,
new String[] { "Name", "Address", "Namespace", "Symbol Type", "Flags", "Locator Hash", // StringField.INSTANCE, StringField.INSTANCE },
"Primary", "Datatype", "Variable Offset", "ExtOrigImportName", "ExtProgAddr", "Comment", // new String[] { "Name", "Address", "Namespace", "Symbol Type", "Flags", "Locator Hash",
"LibPath" }, // "Primary", "Datatype", "Variable Offset", "ExtOrigImportName", "ExtProgAddr", "Comment",
new int[] { SYMBOL_HASH_COL, SYMBOL_PRIMARY_COL, SYMBOL_DATATYPE_COL, SYMBOL_VAROFFSET_COL, // "LibPath" },
SYMBOL_ORIGINAL_IMPORTED_NAME_COL, SYMBOL_EXTERNAL_PROG_ADDR_COL, SYMBOL_COMMENT_COL, // new int[] { SYMBOL_HASH_COL, SYMBOL_PRIMARY_COL, SYMBOL_DATATYPE_COL, SYMBOL_VAROFFSET_COL,
SYMBOL_LIBPATH_COL }); // SYMBOL_ORIGINAL_IMPORTED_NAME_COL, SYMBOL_EXTERNAL_PROG_ADDR_COL, SYMBOL_COMMENT_COL,
// SYMBOL_LIBPATH_COL });
static final int V4_SYMBOL_NAME_COL = 0;
static final int V4_SYMBOL_ADDR_COL = 1;
static final int V4_SYMBOL_PARENT_ID_COL = 2;
static final int V4_SYMBOL_TYPE_COL = 3;
static final int V4_SYMBOL_FLAGS_COL = 4;
// Sparse fields - the following fields are not always applicable so they are optional and
// don't consume space in the database if they aren't used.
static final int V4_SYMBOL_HASH_COL = 5;
static final int V4_SYMBOL_PRIMARY_COL = 6;
static final int V4_SYMBOL_DATATYPE_COL = 7;
static final int V4_SYMBOL_VAROFFSET_COL = 8;
static final int V4_SYMBOL_ORIGINAL_IMPORTED_NAME_COL = 9;
static final int V4_SYMBOL_EXTERNAL_PROG_ADDR_COL = 10;
static final int V4_SYMBOL_COMMENT_COL = 11;
static final int V4_SYMBOL_LIBPATH_COL = 12;
private Table symbolTable; private Table symbolTable;
private AddressMap addrMap; private AddressMap addrMap;
SymbolDatabaseAdapterV4(DBHandle handle, AddressMap addrMap, boolean create) SymbolDatabaseAdapterV4(DBHandle handle, AddressMap addrMap) throws VersionException {
throws VersionException, IOException {
this.addrMap = addrMap; this.addrMap = addrMap;
if (create) { symbolTable = handle.getTable(SYMBOL_TABLE_NAME);
symbolTable = handle.createTable(SYMBOL_TABLE_NAME, SYMBOL_SCHEMA, if (symbolTable == null) {
new int[] { SYMBOL_ADDR_COL, SYMBOL_NAME_COL, SYMBOL_PARENT_ID_COL, SYMBOL_HASH_COL, throw new VersionException("Missing Table: " + SYMBOL_TABLE_NAME);
SYMBOL_PRIMARY_COL, SYMBOL_ORIGINAL_IMPORTED_NAME_COL,
SYMBOL_EXTERNAL_PROG_ADDR_COL });
} }
else { if (symbolTable.getSchema().getVersion() != SYMBOL_VERSION) {
symbolTable = handle.getTable(SYMBOL_TABLE_NAME); int version = symbolTable.getSchema().getVersion();
if (symbolTable == null) { if (version < SYMBOL_VERSION) {
throw new VersionException("Missing Table: " + SYMBOL_TABLE_NAME); throw new VersionException(true);
}
if (symbolTable.getSchema().getVersion() != SYMBOL_VERSION) {
int version = symbolTable.getSchema().getVersion();
if (version < SYMBOL_VERSION) {
throw new VersionException(true);
}
throw new VersionException(VersionException.NEWER_VERSION, false);
} }
throw new VersionException(VersionException.NEWER_VERSION, false);
} }
} }
// @Override
// DBRecord createSymbol(String name, Address address, long namespaceID, SymbolType symbolType,
// SourceType source, boolean isPrimary) throws IOException {
// long nextID = symbolTable.getKey();
//
// // avoiding key 0, as it is reserved for the global namespace
// if (nextID == 0) {
// nextID++;
// }
// return createSymbol(nextID, name, address, namespaceID, symbolType, (byte) source.getStorageId(),
// isPrimary);
// }
@Override @Override
DBRecord createSymbolRecord(String name, long namespaceID, Address address, DBRecord createSymbolRecord(String name, long namespaceID, Address address,
SymbolType symbolType, boolean isPrimary, SourceType source) { SymbolType symbolType, boolean isPrimary, SourceType source) {
throw new UnsupportedOperationException();
long nextID = symbolTable.getKey();
// Avoid key 0, as it is reserved for the global namespace
if (nextID == 0) {
nextID++;
}
long addressKey = addrMap.getKey(address, true);
DBRecord rec = symbolTable.getSchema().createRecord(nextID);
rec.setString(SYMBOL_NAME_COL, name);
rec.setLongValue(SYMBOL_ADDR_COL, addressKey);
rec.setLongValue(SYMBOL_PARENT_ID_COL, namespaceID);
rec.setByteValue(SYMBOL_TYPE_COL, symbolType.getID());
rec.setByteValue(SYMBOL_FLAGS_COL, getSourceTypeFlagsBits(source)); // assume non-pinned
// Sparse columns - these columns don't apply to all symbols.
// they default to null unless specifically set. Null values don't consume space.
rec.setField(SYMBOL_HASH_COL, computeLocatorHash(name, namespaceID, addressKey));
if (isPrimary) {
rec.setLongValue(SYMBOL_PRIMARY_COL, addressKey);
}
return rec;
} }
@Override @Override
void removeSymbol(long symbolID) throws IOException { void removeSymbol(long symbolID) throws IOException {
symbolTable.deleteRecord(symbolID); throw new UnsupportedOperationException();
} }
@Override @Override
@@ -144,7 +112,7 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) { if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return false; return false;
} }
return symbolTable.hasRecord(new LongField(key), SYMBOL_ADDR_COL); return symbolTable.hasRecord(new LongField(key), V4_SYMBOL_ADDR_COL);
} }
@Override @Override
@@ -153,12 +121,12 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) { if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return Field.EMPTY_ARRAY; return Field.EMPTY_ARRAY;
} }
return symbolTable.findRecords(new LongField(key), SYMBOL_ADDR_COL); return symbolTable.findRecords(new LongField(key), V4_SYMBOL_ADDR_COL);
} }
@Override @Override
DBRecord getSymbolRecord(long symbolID) throws IOException { DBRecord getSymbolRecord(long symbolID) throws IOException {
return symbolTable.getRecord(symbolID); return convertV4Record(symbolTable.getRecord(symbolID));
} }
@Override @Override
@@ -168,116 +136,117 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
@Override @Override
RecordIterator getSymbolsByAddress(boolean forward) throws IOException { RecordIterator getSymbolsByAddress(boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, KeyToRecordIterator it = new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, SYMBOL_ADDR_COL, addrMap, forward)); new AddressIndexPrimaryKeyIterator(symbolTable, V4_SYMBOL_ADDR_COL, addrMap, forward));
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException { RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, KeyToRecordIterator it =
SYMBOL_ADDR_COL, addrMap, startAddr, forward)); new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
V4_SYMBOL_ADDR_COL, addrMap, startAddr, forward));
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
void updateSymbolRecord(DBRecord record) throws IOException { void updateSymbolRecord(DBRecord record) throws IOException {
// make sure hash is updated to current name and name space throw new UnsupportedOperationException();
String name = record.getString(SYMBOL_NAME_COL);
long namespaceId = record.getLongValue(SYMBOL_PARENT_ID_COL);
long addressKey = record.getLongValue(SYMBOL_ADDR_COL);
record.setField(SYMBOL_HASH_COL, computeLocatorHash(name, namespaceId, addressKey));
symbolTable.putRecord(record);
} }
@Override @Override
RecordIterator getSymbols() throws IOException { RecordIterator getSymbols() throws IOException {
return symbolTable.iterator(); return new V4ConvertedRecordIterator(symbolTable.iterator());
} }
@Override @Override
RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException { RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, KeyToRecordIterator it =
SYMBOL_ADDR_COL, addrMap, start, end, forward)); new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
V4_SYMBOL_ADDR_COL, addrMap, start, end, forward));
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException { RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, KeyToRecordIterator it =
SYMBOL_ADDR_COL, addrMap, set, forward)); new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
V4_SYMBOL_ADDR_COL, addrMap, set, forward));
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
protected RecordIterator getPrimarySymbols(AddressSetView set, boolean forward) protected RecordIterator getPrimarySymbols(AddressSetView set, boolean forward)
throws IOException { throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable, KeyToRecordIterator it =
SYMBOL_PRIMARY_COL, addrMap, set, forward)); new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
V4_SYMBOL_PRIMARY_COL, addrMap, set, forward));
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
protected DBRecord getPrimarySymbol(Address address) throws IOException { protected DBRecord getPrimarySymbol(Address address) throws IOException {
AddressIndexPrimaryKeyIterator it = new AddressIndexPrimaryKeyIterator(symbolTable, AddressIndexPrimaryKeyIterator it = new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, address, address, true); V4_SYMBOL_PRIMARY_COL, addrMap, address, address, true);
if (it.hasNext()) { if (it.hasNext()) {
return symbolTable.getRecord(it.next()); return convertV4Record(symbolTable.getRecord(it.next()));
} }
return null; return null;
} }
void deleteExternalEntries(Address start, Address end) throws IOException { void deleteExternalEntries(Address start, Address end) throws IOException {
AddressRecordDeleter.deleteRecords(symbolTable, SYMBOL_ADDR_COL, addrMap, start, end, null); throw new UnsupportedOperationException();
} }
@Override @Override
void moveAddress(Address oldAddr, Address newAddr) throws IOException { void moveAddress(Address oldAddr, Address newAddr) throws IOException {
LongField oldKey = new LongField(addrMap.getKey(oldAddr, false)); throw new UnsupportedOperationException();
long newKey = addrMap.getKey(newAddr, true);
Field[] keys = symbolTable.findRecords(oldKey, SYMBOL_ADDR_COL);
for (Field key : keys) {
DBRecord rec = symbolTable.getRecord(key);
rec.setLongValue(SYMBOL_ADDR_COL, newKey);
symbolTable.putRecord(rec);
}
} }
@Override @Override
Set<Address> deleteAddressRange(Address startAddr, Address endAddr, TaskMonitor monitor) Set<Address> deleteAddressRange(Address startAddr, Address endAddr, TaskMonitor monitor)
throws CancelledException, IOException { throws CancelledException, IOException {
throw new UnsupportedOperationException();
AnchoredSymbolRecordFilter filter = new AnchoredSymbolRecordFilter();
AddressRecordDeleter.deleteRecords(symbolTable, SYMBOL_ADDR_COL, addrMap, startAddr,
endAddr, filter);
return filter.getAddressesForSkippedRecords();
} }
@Override @Override
RecordIterator getSymbolsByNamespace(long id) throws IOException { RecordIterator getSymbolsByNamespace(long id) throws IOException {
LongField field = new LongField(id); LongField field = new LongField(id);
return symbolTable.indexIterator(SYMBOL_PARENT_ID_COL, field, field, true); RecordIterator it =
symbolTable.indexIterator(V4_SYMBOL_PARENT_ID_COL, field, field, true);
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator getSymbolsByName(String name) throws IOException { RecordIterator getSymbolsByName(String name) throws IOException {
StringField field = new StringField(name); StringField field = new StringField(name);
return symbolTable.indexIterator(SYMBOL_NAME_COL, field, field, true); RecordIterator it = symbolTable.indexIterator(V4_SYMBOL_NAME_COL, field, field, true);
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator scanSymbolsByName(String startName) throws IOException { RecordIterator scanSymbolsByName(String startName) throws IOException {
StringField field = new StringField(startName); StringField field = new StringField(startName);
return symbolTable.indexIterator(SYMBOL_NAME_COL, field, null, true); RecordIterator it = symbolTable.indexIterator(V4_SYMBOL_NAME_COL, field, null, true);
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator getExternalSymbolsByOriginalImportName(String extLabel) throws IOException { RecordIterator getExternalSymbolsByOriginalImportName(String extLabel) throws IOException {
StringField extLabelField = new StringField(extLabel); StringField extLabelField = new StringField(extLabel);
return symbolTable.indexIterator(SYMBOL_ORIGINAL_IMPORTED_NAME_COL, extLabelField, RecordIterator it =
symbolTable.indexIterator(V4_SYMBOL_ORIGINAL_IMPORTED_NAME_COL, extLabelField,
extLabelField, true); extLabelField, true);
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
RecordIterator getExternalSymbolsByMemoryAddress(Address extProgAddr) throws IOException { RecordIterator getExternalSymbolsByMemoryAddress(Address extProgAddr) throws IOException {
StringField addrField = new StringField(extProgAddr.toString()); StringField addrField = new StringField(extProgAddr.toString());
return symbolTable.indexIterator(SYMBOL_EXTERNAL_PROG_ADDR_COL, addrField, addrField, true); RecordIterator it =
symbolTable.indexIterator(V4_SYMBOL_EXTERNAL_PROG_ADDR_COL, addrField, addrField,
true);
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
@@ -291,8 +260,9 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
Field end = computeLocatorHash(name, id, MAX_ADDRESS_OFFSET); Field end = computeLocatorHash(name, id, MAX_ADDRESS_OFFSET);
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, start, end, true); RecordIterator it = symbolTable.indexIterator(V4_SYMBOL_HASH_COL, start, end, true);
return getNameAndNamespaceFilterIterator(name, id, it); it = getNameAndNamespaceFilterIterator(name, id, it);
return new V4ConvertedRecordIterator(it);
} }
@Override @Override
@@ -302,11 +272,11 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
if (search == null) { if (search == null) {
return null; return null;
} }
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, search, search, true); RecordIterator it = symbolTable.indexIterator(V4_SYMBOL_HASH_COL, search, search, true);
RecordIterator filtered = RecordIterator filtered =
getNameNamespaceAddressFilterIterator(name, namespaceId, addressKey, it); getNameNamespaceAddressFilterIterator(name, namespaceId, addressKey, it);
if (filtered.hasNext()) { if (filtered.hasNext()) {
return filtered.next(); return convertV4Record(filtered.next());
} }
return null; return null;
} }
@@ -315,7 +285,7 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
Address getMaxSymbolAddress(AddressSpace space) throws IOException { Address getMaxSymbolAddress(AddressSpace space) throws IOException {
if (space.isMemorySpace()) { if (space.isMemorySpace()) {
AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable, AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false); V4_SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
if (addressKeyIterator.hasNext()) { if (addressKeyIterator.hasNext()) {
return addrMap.decodeAddress(addressKeyIterator.next()); return addrMap.decodeAddress(addressKeyIterator.next());
} }
@@ -323,7 +293,7 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
else { else {
LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false)); LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false));
DBFieldIterator iterator = DBFieldIterator iterator =
symbolTable.indexFieldIterator(null, max, false, SYMBOL_ADDR_COL); symbolTable.indexFieldIterator(null, max, false, V4_SYMBOL_ADDR_COL);
if (iterator.hasPrevious()) { if (iterator.hasPrevious()) {
LongField val = (LongField) iterator.previous(); LongField val = (LongField) iterator.previous();
Address addr = addrMap.decodeAddress(val.getLongValue()); Address addr = addrMap.decodeAddress(val.getLongValue());
@@ -340,24 +310,82 @@ class SymbolDatabaseAdapterV4 extends SymbolDatabaseAdapter {
return symbolTable; return symbolTable;
} }
private class AnchoredSymbolRecordFilter implements RecordFilter { /**
private Set<Address> set = new HashSet<Address>(); * Returns a record matching the current database schema from the version 4 record.
* @param record the record matching the version 4 schema.
* @return a current symbol record.
*/
private DBRecord convertV4Record(DBRecord record) {
if (record == null) {
return null;
}
DBRecord rec = SymbolDatabaseAdapter.SYMBOL_SCHEMA.createRecord(record.getKey());
@Override String symbolName = record.getString(V4_SYMBOL_NAME_COL);
public boolean matches(DBRecord record) { rec.setString(SymbolDatabaseAdapter.SYMBOL_NAME_COL, symbolName);
// only move symbols whose anchor flag is not on
Address addr = addrMap.decodeAddress(record.getLongValue(SYMBOL_ADDR_COL)); long symbolAddrKey = record.getLongValue(V4_SYMBOL_ADDR_COL);
byte flags = record.getByteValue(SymbolDatabaseAdapter.SYMBOL_FLAGS_COL); rec.setLongValue(SymbolDatabaseAdapter.SYMBOL_ADDR_COL, symbolAddrKey);
boolean pinned = (flags & SymbolDatabaseAdapter.SYMBOL_PINNED_FLAG) != 0;
if (!pinned) { long namespaceId = record.getLongValue(V4_SYMBOL_PARENT_ID_COL);
return true; rec.setLongValue(SymbolDatabaseAdapter.SYMBOL_PARENT_ID_COL, namespaceId);
}
set.add(addr); byte symbolTypeId = record.getByteValue(V4_SYMBOL_TYPE_COL);
return false; rec.setByteValue(SymbolDatabaseAdapter.SYMBOL_TYPE_COL, symbolTypeId);
rec.setByteValue(SymbolDatabaseAdapter.SYMBOL_FLAGS_COL,
record.getByteValue(V4_SYMBOL_FLAGS_COL));
//
// Convert sparse columns
//
if (symbolTypeId == SYMBOL_TYPE_LABEL || symbolTypeId == SYMBOL_TYPE_FUNCTION) {
record.setString(SYMBOL_EXTERNAL_PROG_ADDR_COL,
record.getString(V4_SYMBOL_EXTERNAL_PROG_ADDR_COL));
record.setString(SYMBOL_ORIGINAL_IMPORTED_NAME_COL,
record.getString(V4_SYMBOL_ORIGINAL_IMPORTED_NAME_COL));
}
else if (symbolTypeId == SYMBOL_TYPE_LOCAL_VAR || symbolTypeId == SYMBOL_TYPE_PARAMETER) {
record.setString(SYMBOL_COMMENT_COL, record.getString(V4_SYMBOL_COMMENT_COL));
}
else if (symbolTypeId == SYMBOL_TYPE_LIBRARY) {
// NOTE: don't set new sparse ordinal column
record.setString(SYMBOL_LIBPATH_COL, record.getString(V4_SYMBOL_LIBPATH_COL));
} }
Set<Address> getAddressesForSkippedRecords() { Field hash = record.getFieldValue(V4_SYMBOL_HASH_COL);
return set; if (hash != null) {
rec.setField(SymbolDatabaseAdapter.SYMBOL_HASH_COL, hash);
}
Field primaryAddr = record.getFieldValue(V4_SYMBOL_PRIMARY_COL);
if (primaryAddr != null) {
rec.setField(SymbolDatabaseAdapter.SYMBOL_PRIMARY_COL, primaryAddr);
}
Field dataTypeId = record.getFieldValue(V4_SYMBOL_DATATYPE_COL);
if (dataTypeId != null) {
rec.setField(SymbolDatabaseAdapter.SYMBOL_DATATYPE_COL, dataTypeId);
}
Field varOffset = record.getFieldValue(V4_SYMBOL_VAROFFSET_COL);
if (varOffset != null) {
rec.setField(SymbolDatabaseAdapter.SYMBOL_VAROFFSET_COL, varOffset);
}
return rec;
}
private class V4ConvertedRecordIterator extends ConvertedRecordIterator {
V4ConvertedRecordIterator(RecordIterator originalIterator) {
super(originalIterator, false);
}
@Override
protected DBRecord convertRecord(DBRecord record) {
return convertV4Record(record);
} }
} }
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolDatabaseAdapterV5.java
+351
View File
@@ -0,0 +1,351 @@
/* ###
* IP: GHIDRA
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ghidra.program.database.symbol;
import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import db.*;
import ghidra.program.database.map.*;
import ghidra.program.database.util.EmptyRecordIterator;
import ghidra.program.database.util.RecordFilter;
import ghidra.program.model.address.*;
import ghidra.program.model.symbol.SourceType;
import ghidra.program.model.symbol.SymbolType;
import ghidra.util.exception.CancelledException;
import ghidra.util.exception.VersionException;
import ghidra.util.task.TaskMonitor;
/**
* SymbolDatabaseAdapter for version 5
*
* This version added additional sparse columns to store optional data specific to certain
* symbol types. The adhoc string data column was eliminated.
*/
class SymbolDatabaseAdapterV5 extends SymbolDatabaseAdapter {
static final int SYMBOL_VERSION = 5;
// Used to create a range when searching symbols by name/namespace but don't care about address
private static final long MIN_ADDRESS_OFFSET = 0;
private static final long MAX_ADDRESS_OFFSET = -1;
// NOTE: the primary field duplicates the symbol's address when the symbol is primary. This
// allows us to index this field and quickly find the primary symbols. The field is sparse
// so that non-primary symbols don't consume any space for this field.
static final Schema V5_SYMBOL_SCHEMA = new Schema(SYMBOL_VERSION, "Key",
new Field[] { StringField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
ByteField.INSTANCE, ByteField.INSTANCE, LongField.INSTANCE, LongField.INSTANCE,
LongField.INSTANCE, IntField.INSTANCE, StringField.INSTANCE, StringField.INSTANCE,
StringField.INSTANCE, StringField.INSTANCE, IntField.INSTANCE },
new String[] { "Name", "Address", "Namespace", "Symbol Type", "Flags", "Locator Hash",
"Primary", "Datatype", "Variable Offset", "ExtOrigImportName", "ExtProgAddr", "Comment",
"LibPath", "LibOrdinal" },
new int[] { SYMBOL_HASH_COL, SYMBOL_PRIMARY_COL, SYMBOL_DATATYPE_COL, SYMBOL_VAROFFSET_COL,
SYMBOL_ORIGINAL_IMPORTED_NAME_COL, SYMBOL_EXTERNAL_PROG_ADDR_COL, SYMBOL_COMMENT_COL,
SYMBOL_LIBPATH_COL, SYMBOL_LIB_ORDINAL_COL });
private Table symbolTable;
private AddressMap addrMap;
SymbolDatabaseAdapterV5(DBHandle handle, AddressMap addrMap, boolean create)
throws VersionException, IOException {
this.addrMap = addrMap;
if (create) {
symbolTable = handle.createTable(SYMBOL_TABLE_NAME, SYMBOL_SCHEMA,
new int[] { SYMBOL_ADDR_COL, SYMBOL_NAME_COL, SYMBOL_PARENT_ID_COL, SYMBOL_HASH_COL,
SYMBOL_PRIMARY_COL, SYMBOL_ORIGINAL_IMPORTED_NAME_COL,
SYMBOL_EXTERNAL_PROG_ADDR_COL });
}
else {
symbolTable = handle.getTable(SYMBOL_TABLE_NAME);
if (symbolTable == null) {
throw new VersionException("Missing Table: " + SYMBOL_TABLE_NAME);
}
if (symbolTable.getSchema().getVersion() != SYMBOL_VERSION) {
int version = symbolTable.getSchema().getVersion();
if (version < SYMBOL_VERSION) {
throw new VersionException(true);
}
throw new VersionException(VersionException.NEWER_VERSION, false);
}
}
}
@Override
DBRecord createSymbolRecord(String name, long namespaceID, Address address,
SymbolType symbolType, boolean isPrimary, SourceType source) {
long nextID = symbolTable.getKey();
// Avoid key 0, as it is reserved for the global namespace
if (nextID == 0) {
nextID++;
}
long addressKey = addrMap.getKey(address, true);
DBRecord rec = symbolTable.getSchema().createRecord(nextID);
rec.setString(SYMBOL_NAME_COL, name);
rec.setLongValue(SYMBOL_ADDR_COL, addressKey);
rec.setLongValue(SYMBOL_PARENT_ID_COL, namespaceID);
rec.setByteValue(SYMBOL_TYPE_COL, symbolType.getID());
rec.setByteValue(SYMBOL_FLAGS_COL, getSourceTypeFlagsBits(source)); // assume non-pinned
// Sparse columns - these columns don't apply to all symbols.
// they default to null unless specifically set. Null values don't consume space.
rec.setField(SYMBOL_HASH_COL, computeLocatorHash(name, namespaceID, addressKey));
if (isPrimary) {
rec.setLongValue(SYMBOL_PRIMARY_COL, addressKey);
}
return rec;
}
@Override
void removeSymbol(long symbolID) throws IOException {
symbolTable.deleteRecord(symbolID);
}
@Override
boolean hasSymbol(Address addr) throws IOException {
long key = addrMap.getKey(addr, false);
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return false;
}
return symbolTable.hasRecord(new LongField(key), SYMBOL_ADDR_COL);
}
@Override
Field[] getSymbolIDs(Address addr) throws IOException {
long key = addrMap.getKey(addr, false);
if (key == AddressMap.INVALID_ADDRESS_KEY && !addr.equals(Address.NO_ADDRESS)) {
return Field.EMPTY_ARRAY;
}
return symbolTable.findRecords(new LongField(key), SYMBOL_ADDR_COL);
}
@Override
DBRecord getSymbolRecord(long symbolID) throws IOException {
return symbolTable.getRecord(symbolID);
}
@Override
int getSymbolCount() {
return symbolTable.getRecordCount();
}
@Override
RecordIterator getSymbolsByAddress(boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable,
new AddressIndexPrimaryKeyIterator(symbolTable, SYMBOL_ADDR_COL, addrMap, forward));
}
@Override
RecordIterator getSymbolsByAddress(Address startAddr, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, startAddr, forward));
}
@Override
void updateSymbolRecord(DBRecord record) throws IOException {
// make sure hash is updated to current name and name space
String name = record.getString(SYMBOL_NAME_COL);
long namespaceId = record.getLongValue(SYMBOL_PARENT_ID_COL);
long addressKey = record.getLongValue(SYMBOL_ADDR_COL);
record.setField(SYMBOL_HASH_COL, computeLocatorHash(name, namespaceId, addressKey));
symbolTable.putRecord(record);
}
@Override
RecordIterator getSymbols() throws IOException {
return symbolTable.iterator();
}
@Override
RecordIterator getSymbols(Address start, Address end, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, start, end, forward));
}
@Override
RecordIterator getSymbols(AddressSetView set, boolean forward) throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, set, forward));
}
@Override
protected RecordIterator getPrimarySymbols(AddressSetView set, boolean forward)
throws IOException {
return new KeyToRecordIterator(symbolTable, new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, set, forward));
}
@Override
protected DBRecord getPrimarySymbol(Address address) throws IOException {
AddressIndexPrimaryKeyIterator it = new AddressIndexPrimaryKeyIterator(symbolTable,
SYMBOL_PRIMARY_COL, addrMap, address, address, true);
if (it.hasNext()) {
return symbolTable.getRecord(it.next());
}
return null;
}
void deleteExternalEntries(Address start, Address end) throws IOException {
AddressRecordDeleter.deleteRecords(symbolTable, SYMBOL_ADDR_COL, addrMap, start, end, null);
}
@Override
void moveAddress(Address oldAddr, Address newAddr) throws IOException {
LongField oldKey = new LongField(addrMap.getKey(oldAddr, false));
long newKey = addrMap.getKey(newAddr, true);
Field[] keys = symbolTable.findRecords(oldKey, SYMBOL_ADDR_COL);
for (Field key : keys) {
DBRecord rec = symbolTable.getRecord(key);
rec.setLongValue(SYMBOL_ADDR_COL, newKey);
symbolTable.putRecord(rec);
}
}
@Override
Set<Address> deleteAddressRange(Address startAddr, Address endAddr, TaskMonitor monitor)
throws CancelledException, IOException {
AnchoredSymbolRecordFilter filter = new AnchoredSymbolRecordFilter();
AddressRecordDeleter.deleteRecords(symbolTable, SYMBOL_ADDR_COL, addrMap, startAddr,
endAddr, filter);
return filter.getAddressesForSkippedRecords();
}
@Override
RecordIterator getSymbolsByNamespace(long id) throws IOException {
LongField field = new LongField(id);
return symbolTable.indexIterator(SYMBOL_PARENT_ID_COL, field, field, true);
}
@Override
RecordIterator getSymbolsByName(String name) throws IOException {
StringField field = new StringField(name);
return symbolTable.indexIterator(SYMBOL_NAME_COL, field, field, true);
}
@Override
RecordIterator scanSymbolsByName(String startName) throws IOException {
StringField field = new StringField(startName);
return symbolTable.indexIterator(SYMBOL_NAME_COL, field, null, true);
}
@Override
RecordIterator getExternalSymbolsByOriginalImportName(String extLabel) throws IOException {
StringField extLabelField = new StringField(extLabel);
return symbolTable.indexIterator(SYMBOL_ORIGINAL_IMPORTED_NAME_COL, extLabelField,
extLabelField, true);
}
@Override
RecordIterator getExternalSymbolsByMemoryAddress(Address extProgAddr) throws IOException {
StringField addrField = new StringField(extProgAddr.toString());
return symbolTable.indexIterator(SYMBOL_EXTERNAL_PROG_ADDR_COL, addrField, addrField, true);
}
@Override
RecordIterator getSymbolsByNameAndNamespace(String name, long id) throws IOException {
// create a range of hash fields for all symbols with this name and namespace id over all
// possible addresses
Field start = computeLocatorHash(name, id, MIN_ADDRESS_OFFSET);
if (start == null) {
return EmptyRecordIterator.INSTANCE;
}
Field end = computeLocatorHash(name, id, MAX_ADDRESS_OFFSET);
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, start, end, true);
return getNameAndNamespaceFilterIterator(name, id, it);
}
@Override
DBRecord getSymbolRecord(Address address, String name, long namespaceId) throws IOException {
long addressKey = addrMap.getKey(address, false);
Field search = computeLocatorHash(name, namespaceId, addressKey);
if (search == null) {
return null;
}
RecordIterator it = symbolTable.indexIterator(SYMBOL_HASH_COL, search, search, true);
RecordIterator filtered =
getNameNamespaceAddressFilterIterator(name, namespaceId, addressKey, it);
if (filtered.hasNext()) {
return filtered.next();
}
return null;
}
@Override
Address getMaxSymbolAddress(AddressSpace space) throws IOException {
if (space.isMemorySpace()) {
AddressIndexKeyIterator addressKeyIterator = new AddressIndexKeyIterator(symbolTable,
SYMBOL_ADDR_COL, addrMap, space.getMinAddress(), space.getMaxAddress(), false);
if (addressKeyIterator.hasNext()) {
return addrMap.decodeAddress(addressKeyIterator.next());
}
}
else {
LongField max = new LongField(addrMap.getKey(space.getMaxAddress(), false));
DBFieldIterator iterator =
symbolTable.indexFieldIterator(null, max, false, SYMBOL_ADDR_COL);
if (iterator.hasPrevious()) {
LongField val = (LongField) iterator.previous();
Address addr = addrMap.decodeAddress(val.getLongValue());
if (space.equals(addr.getAddressSpace())) {
return addr;
}
}
}
return null;
}
@Override
Table getTable() {
return symbolTable;
}
private class AnchoredSymbolRecordFilter implements RecordFilter {
private Set<Address> set = new HashSet<Address>();
@Override
public boolean matches(DBRecord record) {
// only move symbols whose anchor flag is not on
Address addr = addrMap.decodeAddress(record.getLongValue(SYMBOL_ADDR_COL));
byte flags = record.getByteValue(SymbolDatabaseAdapter.SYMBOL_FLAGS_COL);
boolean pinned = (flags & SymbolDatabaseAdapter.SYMBOL_PINNED_FLAG) != 0;
if (!pinned) {
return true;
}
set.add(addr);
return false;
}
Set<Address> getAddressesForSkippedRecords() {
return set;
}
}
}
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/database/symbol/SymbolManager.java
+173 -16
View File
@@ -72,6 +72,8 @@ public class SymbolManager implements SymbolTable, ManagerDB {
private NamespaceManager namespaceMgr; private NamespaceManager namespaceMgr;
private VariableStorageManagerDB variableStorageMgr; private VariableStorageManagerDB variableStorageMgr;
private List<LibrarySymbol> libSymbolList; // unmodifiable; use getLibrarySymbolList() to obtain instance
private OldVariableStorageManagerDB oldVariableStorageMgr; // required for upgrade private OldVariableStorageManagerDB oldVariableStorageMgr; // required for upgrade
private AddressMapImpl dynamicSymbolAddressMap; private AddressMapImpl dynamicSymbolAddressMap;
@@ -182,6 +184,10 @@ public class SymbolManager implements SymbolTable, ManagerDB {
// older than program version 10 // older than program version 10
processOldVariableAddresses(monitor); processOldVariableAddresses(monitor);
} }
if (currentRevision < ProgramDB.LIBRARY_ORDINAL_ASSIGNMENT_ADDED_VERSION) {
assignLibraryOrdinals();
}
} }
} }
@@ -663,9 +669,6 @@ public class SymbolManager implements SymbolTable, ManagerDB {
SymbolType symType = sym.getSymbolType(); SymbolType symType = sym.getSymbolType();
try { try {
Address address = sym.getAddress(); Address address = sym.getAddress();
// if (address.isVariableAddress()) {
// variableStorageMgr.deleteVariableStorage(address);
// }
String name = sym.getName(); String name = sym.getName();
boolean primary = sym.isPrimary(); boolean primary = sym.isPrimary();
@@ -674,7 +677,6 @@ public class SymbolManager implements SymbolTable, ManagerDB {
adapter.removeSymbol(id); adapter.removeSymbol(id);
cache.delete(id); cache.delete(id);
//sym.setInvalid(); // already invalidated by removeObj
// if any symbols still exist here, then // if any symbols still exist here, then
// make one of these remaining symbols 'primary' // make one of these remaining symbols 'primary'
@@ -734,11 +736,10 @@ public class SymbolManager implements SymbolTable, ManagerDB {
private SymbolDB getDynamicSymbol(Address memoryAddress) { private SymbolDB getDynamicSymbol(Address memoryAddress) {
long symbolID = getDynamicSymbolID(memoryAddress); long symbolID = getDynamicSymbolID(memoryAddress);
// retrieve the symbol from the cache without validating or refreshing it. We know // Retrieve the symbol from the cache without validating or refreshing it.
// if it exists is should
CodeSymbol symbol = (CodeSymbol) cache.getRaw(symbolID); CodeSymbol symbol = (CodeSymbol) cache.getRaw(symbolID);
if (symbol != null) { if (symbol != null) {
// we know if we got here, the symbol is valid regardless of what it thinks // We know if we got here the symbol is valid, so force it to be valid
symbol.setIsValid(); symbol.setIsValid();
return symbol; return symbol;
} }
@@ -1378,8 +1379,8 @@ public class SymbolManager implements SymbolTable, ManagerDB {
@Override @Override
public LabelHistory[] getLabelHistory(Address addr) { public LabelHistory[] getLabelHistory(Address addr) {
ArrayList<LabelHistory> list = new ArrayList<>(); try (Closeable c = lock.read()) {
try { ArrayList<LabelHistory> list = new ArrayList<>();
RecordIterator iter = historyAdapter.getRecordsByAddress(addrMap.getKey(addr, false)); RecordIterator iter = historyAdapter.getRecordsByAddress(addrMap.getKey(addr, false));
while (iter.hasNext()) { while (iter.hasNext()) {
DBRecord rec = iter.next(); DBRecord rec = iter.next();
@@ -1401,8 +1402,9 @@ public class SymbolManager implements SymbolTable, ManagerDB {
@Override @Override
public void invalidateCache(boolean all) { public void invalidateCache(boolean all) {
variableStorageMgr.invalidateCache(all);
try (Closeable c = lock.write()) { try (Closeable c = lock.write()) {
variableStorageMgr.invalidateCache(all);
libSymbolList = null;
cache.invalidate(); cache.invalidate();
dynamicSymbolAddressMap.reconcile(); dynamicSymbolAddressMap.reconcile();
} }
@@ -2739,7 +2741,11 @@ public class SymbolManager implements SymbolTable, ManagerDB {
} }
/** /**
* Create a Library symbol with the specified name and optional pathname * Create a Library symbol with the specified name and optional pathname.
* The new Library symbol will be assigned the next available ordinal at the end of the
* ordered Library symbol sequence. Use {@link LibrarySymbol#setOrdinal(int)} to adjust
* ordinal.
*
* *
* @param name library name * @param name library name
* @param pathname project file path (may be null) * @param pathname project file path (may be null)
@@ -2757,14 +2763,24 @@ public class SymbolManager implements SymbolTable, ManagerDB {
try (Closeable c = lock.write()) { try (Closeable c = lock.write()) {
DBRecord record = doCreateBasicSymbolRecord(name, null, Address.NO_ADDRESS, DBRecord record = doCreateBasicSymbolRecord(name, null, Address.NO_ADDRESS,
SymbolType.LIBRARY, false, source, true); SymbolType.LIBRARY, false, source, true);
LibrarySymbol.setRecordFields(record, pathname);
List<LibrarySymbol> librarySymbolList = new ArrayList<>(getLibrarySymbolList());
int ordinal = librarySymbolList.size();
if (Library.UNKNOWN.equals(name)) {
// UNKNOWN Library always assigned ordinal of 0
ordinal = 0;
}
LibrarySymbol newLibSymbol = new LibrarySymbol(this, record);
LibrarySymbol.setRecordFields(record, ordinal, pathname);
adapter.updateSymbolRecord(record); adapter.updateSymbolRecord(record);
cache.add(newLibSymbol);
LibrarySymbol newSymbol = new LibrarySymbol(this, record); librarySymbolList.add(ordinal, newLibSymbol);
cache.add(newSymbol); assignLibraryOrdinals(librarySymbolList, true);
symbolAdded(newSymbol); symbolAdded(newLibSymbol);
return newSymbol; return newLibSymbol;
} }
catch (IOException e) { catch (IOException e) {
program.dbError(e); // will not return program.dbError(e); // will not return
@@ -2772,6 +2788,139 @@ public class SymbolManager implements SymbolTable, ManagerDB {
} }
} }
/**
* {@return computed ordinal of the specified library symbol}
* @param libSym library symbol
*/
int computeLibraryOrdinal(LibrarySymbol libSym) {
List<LibrarySymbol> list = getLibrarySymbolList();
for (int i = 0; i < list.size(); i++) {
LibrarySymbol s = list.get(i);
if (s == libSym) {
return i;
}
}
throw new AssertionError("Invalid symbol instance");
}
/**
* @return an ordered unmodifiable list of Library symbols
*/
public List<LibrarySymbol> getLibrarySymbolList() {
List<LibrarySymbol> list = libSymbolList;
if (list == null) {
try (Closeable c = lock.read()) {
list = buildLibrarySymbolList();
libSymbolList = list;
}
catch (IOException e) {
dbError(e);
}
}
return list;
}
private List<LibrarySymbol> buildLibrarySymbolList() throws IOException {
List<LibrarySymbol> list = new ArrayList<>();
byte libraryTypeId = SymbolType.LIBRARY.getID();
RecordIterator iter = new QueryRecordIterator(adapter.getSymbols(),
rec -> libraryTypeId == rec
.getByteValue(SymbolDatabaseAdapter.SYMBOL_TYPE_COL));
// Assume records will be returned in order of symbol ID
long lastId = -1;
while (iter.hasNext()) {
LibrarySymbol libSym = (LibrarySymbol) getSymbol(iter.next());
long id = libSym.getID();
if (id <= lastId) {
throw new AssertionError("Unexpected symbol order");
}
list.add(libSym);
}
Collections.sort(list);
return Collections.unmodifiableList(list);
}
/**
* Assign and store ordinals for all library symbols during upgrade
*/
private void assignLibraryOrdinals() {
List<LibrarySymbol> libSymList = new ArrayList<>(getLibrarySymbolList());
for (int i = 0; i < libSymList.size(); i++) {
LibrarySymbol libSym = libSymList.get(i);
if (Library.UNKNOWN.equals(libSym.getName())) {
// UNKNOWN Library always assigned ordinal of 0 but may have arbitrary
// placement in list prior to upgrade
if (i != 0) {
libSymList.remove(i);
libSymList.add(0, libSym);
}
break;
}
}
assignLibraryOrdinals(libSymList, false);
}
/**
* Update library symbol ordinal to reflect current placement within ordered list.
* The cached {@code libSymbolList} will be updated to refer to the list instance provided.
* @param list new library symbol ordered list
* @param notify if true any library symbol ordinal change will generate change event for
* that symbol.
*/
void assignLibraryOrdinals(List<LibrarySymbol> list, boolean notify) {
for (int i = 0; i < list.size(); i++) {
LibrarySymbol libSym = list.get(i);
libSym.doSetOrdinal(i, notify);
}
libSymbolList = list; // update cached list
}
/**
* Adjust ordinals based upon the movement of an existing Library symbol.
* <p>
* NOTE: The caller is responsible for not displacing UNKNOWN Library if already exists
* at ordinal 0.
*
* @param libSym existing Library symbol
* @param newOrdinal non-negative ordinal (max value will be limited)
*/
void adjustLibraryOrdinals(LibrarySymbol libSym, int newOrdinal) {
if (newOrdinal < 0) {
throw new IllegalArgumentException("Positive ordinal required");
}
List<LibrarySymbol> libSymList = new ArrayList<>(getLibrarySymbolList());
int listSize = libSymList.size();
if (newOrdinal >= listSize) {
newOrdinal = listSize - 1; // do not go beyond end of list
}
int oldOrdinal = libSym.getOrdinal();
if (oldOrdinal == newOrdinal) {
return;
}
LibrarySymbol s = libSymList.remove(oldOrdinal);
if (s != libSym) {
throw new AssertionError("Library symbol ordinal mismatch with ordered list");
}
libSymList.add(newOrdinal, libSym);
assignLibraryOrdinals(libSymList, true);
}
/**
* {@return library symbol at the specified ordinal or null if ordinal is beyond end of list}
* @param ordinal library symbol ordinal
* @throws IndexOutOfBoundsException if a negative ordinal is specified
*/
LibrarySymbol getLibrarySymbolByOrdinal(int ordinal) throws IndexOutOfBoundsException {
List<LibrarySymbol> librarySymbolList = getLibrarySymbolList();
if (ordinal >= librarySymbolList.size()) {
return null;
}
return librarySymbolList.get(ordinal);
}
/** /**
* Create a Class symbol with the specified name and parent * Create a Class symbol with the specified name and parent
* *
@@ -2875,6 +3024,11 @@ public class SymbolManager implements SymbolTable, ManagerDB {
source = validateSource(source, name, address, symbolType); source = validateSource(source, name, address, symbolType);
name = validateName(name, source); name = validateName(name, source);
if ((symbolType == SymbolType.CLASS || symbolType == SymbolType.NAMESPACE) &&
Library.UNKNOWN.equals(name)) {
throw new InvalidInputException(Library.UNKNOWN + " is a reserved Library name");
}
if (checkForDuplicates) { if (checkForDuplicates) {
checkDuplicateSymbolName(address, name, parent, symbolType); checkDuplicateSymbolName(address, name, parent, symbolType);
} }
@@ -2905,6 +3059,9 @@ public class SymbolManager implements SymbolTable, ManagerDB {
if (!addr.isExternalAddress()) { if (!addr.isExternalAddress()) {
throw new IllegalArgumentException("External address required"); throw new IllegalArgumentException("External address required");
} }
if (!namespace.isExternal()) {
throw new IllegalArgumentException("External namespace required");
}
if (externalProgramAddress != null && !externalProgramAddress.isLoadedMemoryAddress()) { if (externalProgramAddress != null && !externalProgramAddress.isLoadedMemoryAddress()) {
throw new IllegalArgumentException("Memory address required for external program"); throw new IllegalArgumentException("Memory address required for external program");
} }
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/listing/Library.java
+13 -2
View File
@@ -16,9 +16,10 @@
package ghidra.program.model.listing; package ghidra.program.model.listing;
import ghidra.program.model.symbol.*; import ghidra.program.model.symbol.*;
import ghidra.util.exception.InvalidInputException;
/** /**
* Interface for a Library namespace. * Interface for a Library dependency and namespace.
*/ */
public interface Library extends Namespace { public interface Library extends Namespace {
@@ -30,10 +31,20 @@ public interface Library extends Namespace {
} }
/** /**
* @return the associated program within the project which corresponds to this library * @return the associated program file pathname within the project which corresponds to this library.
*/ */
public String getAssociatedProgramPath(); public String getAssociatedProgramPath();
/**
* Sets the program file pathname within the project which corresponds to this library.
* <p>
* NOTE: Assigning a path to {@link Library#UNKNOWN} Library will be ignored.
*
* @param programPath a program file pathname or null to clear the stored path.
* @throws InvalidInputException on invalid programPath is specified
*/
public void setAssociatedProgramPath(String programPath) throws InvalidInputException;
/** /**
* Get the Library which contains the specified external symbol. * Get the Library which contains the specified external symbol.
* @param symbol external symbol * @param symbol external symbol
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/symbol/ExternalLocation.java
+9
View File
@@ -42,6 +42,15 @@ public interface ExternalLocation {
*/ */
public String getLibraryName(); public String getLibraryName();
/**
* NOTE: If this external location corresponds to a back-reference this
* may correspond to an application and not a library.
*
* @return the external Program path which contains the referenced symbol or
* null if unknown.
*/
public String getExternalLibraryPath();
/** /**
* Returns the parent namespace containing this location. * Returns the parent namespace containing this location.
* @return the parent namespace containing this location. * @return the parent namespace containing this location.
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/symbol/ExternalManager.java
+69 -5
View File
@@ -15,6 +15,7 @@
*/ */
package ghidra.program.model.symbol; package ghidra.program.model.symbol;
import java.util.List;
import java.util.Set; import java.util.Set;
import ghidra.program.model.address.Address; import ghidra.program.model.address.Address;
@@ -31,13 +32,32 @@ import ghidra.util.exception.InvalidInputException;
public interface ExternalManager { public interface ExternalManager {
/** /**
* Returns an array of all external names for which locations have been defined. * Returns an array of all external Library names sorted by preferred search order.
* @return array of external names * This order reflects the preferred search order when looking for external symbols
* which were not linked to a specific Library.
* <p>
* NOTE: The {@link Library#UNKNOWN} library will always be returned as first in the list
* but will not have an associated program path and cannot be searched.
*
* @return array of all external Library names sorted by preferred search order.
*/ */
public String[] getExternalLibraryNames(); public String[] getExternalLibraryNames();
/**
* Get a list of all external Libraries sorted by preferred search order.
* This order reflects the preferred search order when looking for external symbols
* which were not linked to a specific Library.
* <p>
* NOTE: The {@link Library#UNKNOWN} library will always be returned as first in the list
* but will not have an associated program path and cannot be searched.
*
* @return list of all external Libraries sorted by preferred search order.
*/
public List<Library> getLibraries();
/** /**
* Get the Library which corresponds to the specified name * Get the Library which corresponds to the specified name
*
* @param libraryName name of library * @param libraryName name of library
* @return library or null if not found * @return library or null if not found
*/ */
@@ -45,6 +65,7 @@ public interface ExternalManager {
/** /**
* Removes external name if no associated ExternalLocation's exist * Removes external name if no associated ExternalLocation's exist
*
* @param libraryName external library name * @param libraryName external library name
* @return true if removed, false if unable to due to associated locations/references * @return true if removed, false if unable to due to associated locations/references
*/ */
@@ -54,35 +75,69 @@ public interface ExternalManager {
* Returns the file pathname associated with an external name. * Returns the file pathname associated with an external name.
* Null is returned if either the external name does not exist or * Null is returned if either the external name does not exist or
* a pathname has not been set. * a pathname has not been set.
*
* @param libraryName external name * @param libraryName external name
* @return project file pathname or null * @return project file pathname or null
*/ */
public String getExternalLibraryPath(String libraryName); public String getExternalLibraryPath(String libraryName);
/** /**
* Sets the file pathname associated with an existing external name. * Sets the file pathname associated with an external name.
* If the Library namespace/symbol does not already exist it will be created provided
* the libraryName does not conflict with another namespace whose parent is the global
* namespace.
* <p>
* NOTE: Assigning path for {@link Library#UNKNOWN} Library will be ignored.
* <p>
* NOTE: Assigning path to a non-Library namespace will fail silently.
*
* @param libraryName the name of the library to associate with a file. * @param libraryName the name of the library to associate with a file.
* @param pathname the path to the program to be associated with the library name. * @param pathname the path to the program to be associated with the library name.
* @param userDefined true if the external path is being specified by the user * @param userDefined true if the external path is being specified by the user
* @throws InvalidInputException on invalid input * @throws InvalidInputException on invalid input specified
*/ */
public void setExternalPath(String libraryName, String pathname, boolean userDefined) public void setExternalPath(String libraryName, String pathname, boolean userDefined)
throws InvalidInputException; throws InvalidInputException;
/**
* {@return the ordinal associated with an external library which represents its
* sequence within the order list of libraries or -1 if library name not found}
*
* @param libraryName the library name
*/
public int getLibraryOrdinal(String libraryName);
/**
* Sets the Library search ordinal associated with an external name.
* <p>
* Assigning ordinal for {@link Library#UNKNOWN} Library will fail and return -1.
* Assigning ordinal to a non-existing Library will fail and return -1.
* <p>
* NOTE: The actual ordinal applied my be limited based on placement restrictions.
*
* @param libraryName the name of the library to position within Library search sequence
* @param ordinal library ordinal greater or equal to 1
* @return the actual ordinal applied or -1 if change failed.
*/
public int setLibraryOrdinal(String libraryName, int ordinal);
/** /**
* Change the name of an existing external name. * Change the name of an existing external name.
*
* @param oldName the old name of the external library name. * @param oldName the old name of the external library name.
* @param newName the new name of the external library name. * @param newName the new name of the external library name.
* @param source the source of this external library * @param source the source of this external library
* @return true if symbol was found and renamed, false if symbol not found
* @throws DuplicateNameException if name conflicts with another symbol. * @throws DuplicateNameException if name conflicts with another symbol.
* @throws InvalidInputException if an invalid or null name specified (see * @throws InvalidInputException if an invalid or null name specified (see
* {@link SymbolUtilities#validateName}). * {@link SymbolUtilities#validateName}).
*/ */
public void updateExternalLibraryName(String oldName, String newName, SourceType source) public boolean updateExternalLibraryName(String oldName, String newName, SourceType source)
throws DuplicateNameException, InvalidInputException; throws DuplicateNameException, InvalidInputException;
/** /**
* Get an iterator over all external locations associated with the specified Library. * Get an iterator over all external locations associated with the specified Library.
*
* @param libraryName the name of the library to get locations for * @param libraryName the name of the library to get locations for
* @return external location iterator * @return external location iterator
*/ */
@@ -91,6 +146,7 @@ public interface ExternalManager {
/** /**
* Get an iterator over all external locations which have been associated to * Get an iterator over all external locations which have been associated to
* the specified memory address * the specified memory address
*
* @param memoryAddress memory address * @param memoryAddress memory address
* @return external location iterator * @return external location iterator
*/ */
@@ -151,6 +207,7 @@ public interface ExternalManager {
/** /**
* Determines if the indicated external library name is being managed (exists). * Determines if the indicated external library name is being managed (exists).
*
* @param libraryName the external library name * @param libraryName the external library name
* @return true if the name is defined (whether it has a path or not). * @return true if the name is defined (whether it has a path or not).
*/ */
@@ -158,6 +215,7 @@ public interface ExternalManager {
/** /**
* Adds a new external library name * Adds a new external library name
*
* @param libraryName the new external library name to add. * @param libraryName the new external library name to add.
* @param source the source of this external library * @param source the source of this external library
* @return library external {@link Library namespace} * @return library external {@link Library namespace}
@@ -173,6 +231,7 @@ public interface ExternalManager {
* Get or create an external location associated with a library/file named {@code libraryName} * Get or create an external location associated with a library/file named {@code libraryName}
* and the location within that file identified by {@code extLabel} and/or its memory address * and the location within that file identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified. * {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param libraryName the external library name * @param libraryName the external library name
* @param extLabel the external label or null * @param extLabel the external label or null
* @param extAddr the external memory address or null * @param extAddr the external memory address or null
@@ -191,6 +250,7 @@ public interface ExternalManager {
* Create an external location in the indicated external parent namespace * Create an external location in the indicated external parent namespace
* and identified by {@code extLabel} and/or its memory address * and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified. * {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param extNamespace the external namespace * @param extNamespace the external namespace
* @param extLabel the external label or null * @param extLabel the external label or null
* @param extAddr the external memory address or null * @param extAddr the external memory address or null
@@ -208,6 +268,7 @@ public interface ExternalManager {
* Get or create an external location in the indicated external parent namespace * Get or create an external location in the indicated external parent namespace
* and identified by {@code extLabel} and/or its memory address * and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified. * {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param extNamespace the external namespace * @param extNamespace the external namespace
* @param extLabel the external label or null * @param extLabel the external label or null
* @param extAddr the external memory address or null * @param extAddr the external memory address or null
@@ -227,6 +288,7 @@ public interface ExternalManager {
* Create an external {@link Function} in the external {@link Library} namespace * Create an external {@link Function} in the external {@link Library} namespace
* {@code libararyName} and identified by {@code extLabel} and/or its memory address * {@code libararyName} and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified. * {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param libraryName the external library name * @param libraryName the external library name
* @param extLabel label within the external program, may be null if extAddr is not null * @param extLabel label within the external program, may be null if extAddr is not null
* @param extAddr memory address within the external program, may be null * @param extAddr memory address within the external program, may be null
@@ -245,6 +307,7 @@ public interface ExternalManager {
* Create an external {@link Function} in the indicated external parent namespace * Create an external {@link Function} in the indicated external parent namespace
* and identified by {@code extLabel} and/or its memory address * and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified. * {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param extNamespace the external namespace * @param extNamespace the external namespace
* @param extLabel the external label or null * @param extLabel the external label or null
* @param extAddr the external memory address or null * @param extAddr the external memory address or null
@@ -262,6 +325,7 @@ public interface ExternalManager {
* Get or create an external {@link Function} in the indicated external parent namespace * Get or create an external {@link Function} in the indicated external parent namespace
* and identified by {@code extLabel} and/or its memory address * and identified by {@code extLabel} and/or its memory address
* {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified. * {@code extAddr}. Either or both {@code extLabel} or {@code extAddr} must be specified.
*
* @param extNamespace the external namespace * @param extNamespace the external namespace
* @param extLabel the external label or null * @param extLabel the external label or null
* @param extAddr the external memory address or null * @param extAddr the external memory address or null
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/util/ChangeManager.java
-7
View File
@@ -357,13 +357,6 @@ public interface ChangeManager {
@Deprecated @Deprecated
public static final ProgramEvent DOCR_MEM_REF_PRIMARY_REMOVED = REFERENCE_PRIMARY_REMOVED; public static final ProgramEvent DOCR_MEM_REF_PRIMARY_REMOVED = REFERENCE_PRIMARY_REMOVED;
/**
* The external path name changed for an external program name.
* @deprecated Event type numeric constants have been changed to enums. Use the enum directly.
*/
@Deprecated
public static final ProgramEvent DOCR_EXTERNAL_PATH_CHANGED = EXTERNAL_PATH_CHANGED;
/** /**
* An external program name was added. * An external program name was added.
* @deprecated Event type numeric constants have been changed to enums. Use the enum directly. * @deprecated Event type numeric constants have been changed to enums. Use the enum directly.
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/util/ProgramEvent.java
+6 -3
View File
@@ -53,9 +53,12 @@ public enum ProgramEvent implements EventType {
SYMBOL_DATA_CHANGED, // some symbol property was changed SYMBOL_DATA_CHANGED, // some symbol property was changed
SYMBOL_ADDRESS_CHANGED, // the symbol's address changed (only applies to param and variables) SYMBOL_ADDRESS_CHANGED, // the symbol's address changed (only applies to param and variables)
EXTERNAL_ENTRY_ADDED, // an external entry point was added EXTERNAL_ENTRY_ADDED, // an external entry point was added (i.e., Export)
EXTERNAL_ENTRY_REMOVED, // an external entry point was removed EXTERNAL_ENTRY_REMOVED, // an external entry point was removed (i.e., Export)
EXTERNAL_PATH_CHANGED, // the external path name changed for an external program
//
// Events related to Library symbols and associated External Location
//
EXTERNAL_NAME_ADDED, // an external program name was added EXTERNAL_NAME_ADDED, // an external program name was added
EXTERNAL_NAME_REMOVED, // an external program name was removed EXTERNAL_NAME_REMOVED, // an external program name was removed
EXTERNAL_NAME_CHANGED, // the name of an external program was changed EXTERNAL_NAME_CHANGED, // the name of an external program was changed
Ghidra/Test/IntegrationTest/src/screen/java/help/screenshot/ReferencesPluginScreenShots.java
+11 -5
View File
@@ -38,9 +38,9 @@ import ghidra.program.model.listing.CodeUnit;
import ghidra.program.model.listing.Program; import ghidra.program.model.listing.Program;
import ghidra.program.model.mem.Memory; import ghidra.program.model.mem.Memory;
import ghidra.program.model.mem.MemoryAccessException; import ghidra.program.model.mem.MemoryAccessException;
import ghidra.program.model.symbol.ExternalManager;
import ghidra.util.InvalidNameException; import ghidra.util.InvalidNameException;
import ghidra.util.exception.CancelledException; import ghidra.util.exception.*;
import ghidra.util.exception.VersionException;
import ghidra.util.task.TaskMonitor; import ghidra.util.task.TaskMonitor;
public class ReferencesPluginScreenShots extends GhidraScreenShotGenerator { public class ReferencesPluginScreenShots extends GhidraScreenShotGenerator {
@@ -122,9 +122,15 @@ public class ReferencesPluginScreenShots extends GhidraScreenShotGenerator {
} }
@Test @Test
public void testExternal_names_dialog() { public void testExternal_names_dialog() throws InvalidInputException {
showProvider(ExternalReferencesProvider.class); ExternalManager externalManager = program.getExternalManager();
captureProvider(ExternalReferencesProvider.class); program.withTransaction("Set Program Path", () -> {
externalManager.setExternalPath("USER32.DLL", "/libs/user32.dll", true);
});
ExternalReferencesProvider provider = showProvider(ExternalReferencesProvider.class);
JTable table = findComponent(provider.getComponent(), JTable.class);
selectRow(table, 0);
captureIsolatedProvider(ExternalReferencesProvider.class, 500, 500);
} }
@Test @Test