# Optimization Grammar

Each of the macros described in the previous section must conform to the
grammar below. It must be possible to compile the rules using C++
compiler, and parse the rules using an external tool. So, certain
constructs (such as `(int)n` rather than `int(n)`) will work in C++
but are not permitted by the rules.

## Lexical

The only separators used in the grammar: Left and Right parentheses, and
the comma (also, the plus sign is used in only one context):

'('  ')'  ','   '+'
    Copy to clipboard

All other characters must be one of the following:

<string_constant>           "Chars"
    Copy to clipboard

Nothing unprintable, no space or escapes; so `"` and `\` cannot be
used. Empty string is not allowed. As in C/C++, if two or more string
constants appear with nothing (or just white space) separating them,
they are concatenated. However, full C escapes are allowed for strings
used in `MESSAGE`, etc.

<integer_const>            E.g. 1230u
    Copy to clipboard

As in C, including with U or L suffixes; and with optional sign. Use of
an octal `value >=8` (e.g. 010) will generate a warning.

Suffix ‘`L`’ is ignored; ‘`U`’ causes the value to be seen as
‘`size_t`’ rather than ‘`int`’}

<float_const>             E.g.  -1.331e-2
    Copy to clipboard

As in C, and with optional sign; ‘`f`’ suffix allowed.

<name>         [A-Za-z_][A-Za-z_0-9]*
    Copy to clipboard

Any C identifier (or keyword). The names `OK`, true, false are
recognized as bool constants. Also, `INF` and `NEG_INF` are
recognized as float constants representing infinity’ and -infinity.
Throughout the grammar, quoted symbols such as `'ADD'` appear, this
means “a which is ADD”.

<dtype_tag>               DType::<name>
    Copy to clipboard

This represents a constant of type `DType`; the must be one of the
tags of the `DType` enum.

## General

operand_tag ::=
       <string_constant>      // representing an operand tag
    Copy to clipboard

opstring ::=
       <string_constant>      // representing the function of an Op node to be matched or built
    Copy to clipboard

## Top Level

For the external rule-parsing tool, it is required that the
`DEF_PACKAGE_OPTIMIZATION` and the following ‘(’ both appear at the
start of the same source line; otherwise, tokens may be separated by
arbitrary white space, and/or C++ comments.

Anything after the closing ‘)’ of a `DEF_PACKAGE_OPTIMIZATION(...)`,
up to the end of file or the next `DEF_PACKAGE_OPTIMIZATION`, is
ignored; however, the first non-space character following a
`DEF_PACKAGE_OPTIMIZATION(...)` may not be a left-parenthesis or
comma.

optimization_rule ::=
       DEF_PACKAGE_OPTIMIZATION ( 'pass_spec' , 'match_expression ',' constraint_expression ',' replacement_rule ')'
    Copy to clipboard

pass_spec ::=
           <name>
        |  <name> '+' <integer_const>
    Copy to clipboard

Note: in this context, \* the must be one of the predefined names for
pass groups \* the integer constant may only be a decimal constant, with
no sign or suffix

With similar set of rules, user can set the configure a custom Op with
cost and flag as the following:

optimization_rule ::=
       DEF_PACKAGE_OP_AND_COST_AND_FLAGS( 'function_name', 'op_name_as_str', 'cost_of_an_op', 'resources_utilized')
    Copy to clipboard

resources_utilized::=
        IS_CONST            // consant propogation (default flag value)
    |   RESOURCE_HVX,       // utilizing HVX hardware
    
    * User can set more than one flag, separated by comma.
    Copy to clipboard

To simply register an Op, without any cost or resources, use the
following:

DEF_PACKAGE_OP( 'function_name' , 'op_name_as_str')
    Copy to clipboard

## Match

See also ref: `OptMatch`

match_expression ::=
        match_op1
    Copy to clipboard

Note that \* `Op( opstr, inp1, inp2,.. )` matches a specific graph op
with the given list of 0 or more input operands \*
`OpVarIn( opstr, inp1, inp2... )` matches a specific graph op with the
given list of 0 or more input operands and possibly additional input
operands. \* If the same operand tag appears more than once in the match
expression, it indicates that these parts of the matched pattern must
all reference the same node. \* `LET( "opname", ... )` can be wrapped
around any `Op` or `OpVarIn` (other than the root), to attach an
operand tag to the output of that Op. \* Within a given match
expression, a tag used as the first parameter of a `LET` cannot appear
anywhere within the second operand of the same `LET`, and may not be
used as the first parameter of any other `LET`.

match_op1 ::=
           'Op'      '(' opstring [',' match_op]* ')'
        |  'OpVarIn' '(' opstring [',' match_op]* ')'
    Copy to clipboard

match_op2 ::=
           'LET' '(' operand_tag ',' match_op1 ')'
        |  match_op1
    Copy to clipboard

match_op ::=
           operand_tag
       |  match_op2
    Copy to clipboard

## Constraint

Note: \* All of the operand\_tag in constraints must exist in the
corresponding ‘match’ rule, or must be the special tag `"*"`. \* in
general, constraint expressions have types: -
`bool, int, size, float, dtype`

- Currently, ‘dtype’ support is limited to:

    - constants e.g. `DType::Float`
    - comparison to `dtype` using `EQ` and `NE` only;
    - `SELECT( bool, dtype, dtype ) -> dtype`
    - property `DTYPE_OF(param)` gives a `dtype`
    - conversion to int in via `INT()`
    - `WITH_OUTPUT_TYPE(..)` requires a `dtype` expression as its
first parameter.

For more detail on the constraint functions, see ref: `OptConstraint`

constraint_expression ::=
           cst_expr                // must be bool type
    Copy to clipboard

cst_expr ::=
           cst_const
        |  cst_option
        |  cst_oper
        |  cst_property
        |  cst_constval
        |  cst_external
        |  cst_macro
    Copy to clipboard

cst_const ::=
           'OK' |  'true' |  'false'         // type = bool
        |  'INF' | 'NEG_INF'                 // type = float
        |  <integer_const>                   // type = 'int'; or size, if U suffix
        |  <float_const>                     // type = 'float'
        |  <dtype_tag>                       // type = 'dtype'
        | 'LAYOUT_CHUNKSIZE' '(' <name>  ',' <integer_const> ')'  // type = size
                // the <name> is the name of a tensor class.
    Copy to clipboard

cst_option ::=
           'OPTION_INT' '(' "name_of_option" ')'         // type = int
        |  'OPTION_UINT' '(' "name_of_option" ')'        // type = size
        |  'OPTION_BOOL' '(' "name_of_option" ')'        // type = bool
        |  'OPTION_FLOAT' '(' "name_of_option" ')'       // type = float
    Copy to clipboard

Read values from graph options. Options can be read as any type; the value is converted (perhaps with loss).
A ‘bool’ option is read as integer 0 or 1; a ‘string’ option reads as 0 if empty, 1 if not.
OPTION\_BOOL will convert a non-bool type as if by (value!=0).

cst_oper ::=
            typecast '('  cst_expr ')'
        |   'NOT' '(' cst_expr ')'                    // bool -> bool
        |   'NEG' '(' cst_expr ')'
        |   'ABS' '(' cst_expr ')'
        |   'IS_POW2' '(' cst_int ')'                // int, or size -> bool
        |   'SELECT' '('   cst_expr ',' cst_expr  ',' cst_expr ')' // first must be bool
        |   'ROUNDUP' '('  cst_int ',' cst_int ')'      // int/size only; second must be power of 2
        |   reduce_op '('  cst_expr [ ',' cst_expr]* ')'
        |   binary_op '('  cst_expr ',' cst_expr ')'
        |   compare_op '(' cst_expr ',' cst_expr ')'        // result is bool
    Copy to clipboard

typecast ::=
        'UINT' |  'INT' |  'FLOAT' | 'DTYPE'
    Copy to clipboard

reduce_op ::=
            'AND' |  'OR'  |   'XOR'
        |   'ADD' |  'MUL'
        |   'MIN' |  'MAX'
    Copy to clipboard

Note, `REM(a,b)` and `MOD(a,b)` are the same when applied to values
&gt;0; for negative values, `REM(a,b)`, if not zero, has the same sign as
‘a’; `MOD(a,b)`, if not zero, has the same sign as ‘b’.

binary_op ::=
           'SUB' |  'DIV' | 'REM' | 'MOD'
    Copy to clipboard

compare_op ::=
            'EQ' |  'NE'
         |  'LT' |  'GT' |  'LE' |  'GE'
    Copy to clipboard

cst_opref ::=
            operand_tag
        |   'INPUT_OF'  '(' cst_opref ',' cst_int ')'
        |   'OUTPUT_OF'  '(' cst_opref ',' cst_int ')'
        |   'SELECT' '(' cst_expr ',' cst_opref ',' cst_opref ')'  // expr must be bool
    Copy to clipboard

The “`cst_property`” operations below extract properties of the output
of the operand specified by the `cst_opref`.

cst_property ::=
            'RANK_OF' '(' cst_opref ')'
        |   'DIM_OF'  '(' cst_opref ',' cst_int ')'
        |   'STEPSIZE_OF' '(' cst_opref  ')'
        |   'DTYPE_OF' '(' cst_opref  ')'
        |   'ELEMENTSIZE_OF' '(' cst_opref  ')'
        |   'INPUTS_OF' '(' cst_opref ')'
        |   'OUTPUTS_OF' '(' cst_opref ')'
    Copy to clipboard

The operations below give a boolean result and can be used to compare two
operands. `SAME_ENCODING` means the operands have the same DType; and if the
DType is quantized, the two also have the same quantization.

`SAME_OP` means that both operands refer to the same node in the graph. It is
possible for the operands to refer to different nodes which will later be
merged as common sub-expressions; in which case this will return ‘false’

cst_op_compare ::=
            'SAME_ENCODING' '(' cst_opref ',' cst_opref ')'
        |   'SAME_OP'  '(' cst_opref ',' cst_opref ')'
    Copy to clipboard

These operations can extract a scalar value from a `Const` operand at
the given index. In case of failure, i.e. when the the operand is not
‘Const’, or when the index is out of range, `CONSTVAL_INT` returns
`MIN_INT`, and `CONSTVAL_FLOAT` returns `NaN`. `CONSTVAL_INT`
will also fail if the value is not an integer that fits in `int32`.
The corresponding `CONSTVAL_INT_VALID` and `CONSTVAL\_FLOAT\_VALID`
return ‘true’ if the operation will succeed, and ‘false’ if it will not.

cst_constval ::=
            'CONSTVAL_INT'         '(' cst_opref ',' cst_int ')'
        |   'CONSTVAL_INT_VALID'   '(' cst_opref ',' cst_int ')'
        |   'CONSTVAL_FLOAT'       '(' cst_opref ',' cst_int ')'
        |   'CONSTVAL_FLOAT_VALID' '(' cst_opref ',' cst_int ')'
    Copy to clipboard

In the grammar, ‘cst\_int’ generally means a ‘cst\_expr’ of integer or size type.
But, when the expression appears within the last parameter of an `AUTOSPLIT` or  `OP_ITER` , the special
operations listed below may also appear; these obtain one of the variables from the iteration.

cst_int ::=
             cst_expr                // with int or size type
        |   'SPLIT_DIM' '(' split_tag ')'
        |   'SPLIT_START' '(' split_tag ')'
        |   'SPLIT_SIZE' '(' split_tag ')'
        |   'ITER_VAR' '(' split_tag ')'   // synonym for SPLIT_START (use with OP_ITER, INPUT_OF, OUTPUT_OF)
    Copy to clipboard

`EXTERNAL_CONSTRAINT` calls an external C++ function which is expected
to return bool. The first parameter is an “`OperandTag`”; the
remainder (if any) must evaluate to a scalar type
(`int, size_t, float, bool`) matching the parameter types of the
function. You can use general `cst_expr`,
e.g. `EXTERNAL_CONSTRAINT( funcname, "Operand", RANK_OF("Operand"))`

cst_external ::=
        'EXTERNAL_CONSTRAINT' '(' <name> ','  operand_tag [ ',' cst_expr ]* ')'
    Copy to clipboard

‘cst\_macro’ are equivalent to the expansions given:

cst_macro ::=
           'IS_QUINT8' '(' cst_opref ')'         // IS_QUINT8(x) => EQ(DTYPE_OF(x),DType::QUInt8)
        |  'IS_QINT8' '(' cst_opref ')'          // IS_QINT8(x) => EQ(DTYPE_OF(x),DType::QInt8)
        |  'IS_QUINT16' '(' cst_opref ')'        // IS_QUINT16(x) => EQ(DTYPE_OF(x),DType::QUInt16)
        |  'IS_QINT16' '(' cst_opref ')'         // IS_QINT16(x) => EQ(DTYPE_OF(x),DType::QInt16)
        |  'IS_QINT32' '(' cst_opref ')'         // IS_QINT32(x) => EQ(DTYPE_OF(x),DType::QInt32)
        |  'IS_INT32' '(' cst_opref ')'          // IS_INT32(x) => EQ(DTYPE_OF(x),DType::Int32)
        |  'IS_FLOAT16' '(' cst_opref ')'        // IS_FLOAT16(x) => EQ(DTYPE_OF(x),DType::Float16)
        |  'IS_FLOAT32' '(' cst_opref ')'        // IS_FLOAT32(x) => EQ(DTYPE_OF(x),DType::Float32)
        |  'IS_FLOAT' '(' cst_opref ')'          // IS_FLOAT(x) => IS_FLOAT32(x)
        |  'DIM_BATCHES' '(' cst_opref ')'       // DIM_BATCHES(x) => DIM_OF(x,0)
        |  'DIM_HEIGHT' '(' cst_opref ')'        // DIM_HEIGHT(x) => DIM_OF(x,1)
        |  'DIM_WIDTH' '(' cst_opref ')'         // DIM_WIDTH(x) => DIM_OF(x,2)
        |  'DIM_DEPTH' '(' cst_opref ')'         // DIM_DEPTH(x) => DIM_OF(x,3)
        |  'DIM_FILTHEIGHT' '(' cst_opref ')'    // DIM_FILTHEIGHT(x) => DIM_OF(x,0)
        |  'DIM_FILTWIDTH' '(' cst_opref ')'     // DIM_FILTWIDTH(x) => DIM_OF(x,1)
        |  'DIM_FILTDEPTH' '(' cst_opref ')'     // DIM_FILTDEPTH(x) => DIM_OF(x,2)
        |  'DIM_NFILTS' '(' cst_opref ')'        // DIM_NFILTS(x) => DIM_OF(x,3)
        |  'SAME_SHAPE' '(' cst_opref ','  cst_opref ')'
                // SAME_SHAPE(x,y) => AND( EQ( DIM_OF(x,3), DIM_OF(y,3)), EQ( DIM_OF(x,2), DIM_OF(y,2)),
                //                        EQ( DIM_OF(x,1), DIM_OF(y,1)), EQ( DIM_OF(x,0), DIM_OF(y,0)))
    Copy to clipboard

## Replacement

Notes: \* Some of the entities in the Replacement Grammar refer to
`cst_expr` from the Constraint grammar \* All of the operand\_tag must
exist in the corresponding ‘match’ rule, or must be the special tag
`"*"`. \* Some of the entities have a ‘`split_tag`’ operand, which
names a split context. The scope of these names is the entire
replacement rule containing them.

The following apply in a well-formed rule: \* Each instance of
`AUTOSPLIT` or `OP_ITER` in a rule must have a distinct split\_tag as its second
operand (usually there is at most one, and the tag is `"I"`). \* Every
other entity referring to a split\_tag must be contained within the last
operand of an `AUTOSPLIT` or `OP_ITER` entity which has the same split\_tag in its
second operand. \* An `AUTOSPLIT` must contain, within its third
operand, at least one such entity referencing the same split\_tag. \*
more detailed checks on the validity of the overall construct could be
defined.

For more detail on the replacement operations, see ref:
`OptReplacement`

split_tag ::=
       <string_constant>      // representing a split context
    Copy to clipboard

replacement_rule ::=
        repl_op
    Copy to clipboard

Note that ‘Operand’ is redundant : whenever a string constant appears in
a ‘`repl_op`’ context, it is assumed to be an operand tag and
‘Operand’ is applied.

repl_op ::=
             cst_opref
        |    'Operand'  '('  operand_tag ')'
        |    'Op'  '(' opstring [',' repl_op]* ')'
        |    'WrapOp'  '(' opstring ',' repl_op ')'
        |    'WrapOpAlways'  '(' opstring ',' repl_op ')'
        |    'gen_Shape' '(' cst_int [',' cst_int ]* ')'
        |    'gen_ShapeOf' '(' cst_opref ')'
        |    'gen_ConstScalar_f32' '(' cst_expr ')'
        |    'gen_ConstScalar_i32' '(' cst_int ')'
        |    'gen_ConstArr_f32' '(' cst_expr ',' cst_int')      // gen_ConstArr_f32(floatval, n) -> array[1,1,1,n] filled with floatval
        |    'gen_ConstArr_i32' '(' cst_expr ',' cst_int')      // gen_ConstArr_i32(intval, n) -> array[1,1,1,n] filled with intval
        |    'gen_ConstArr_vals_i32' '(' cst_int [',' cst_int]* ')'
                     // gen_ConstArr_vals_i32(x,y,z,..) -> array [1,1,1,n] of int filled with x,y,z ...
        |    'AUTOSPLIT' '(' cst_int ',' split_tag ',' cst_int ',' repl_op ')'
        |    'OP_ITER' '(' repl_op ',' split_tag ',' cst_int ',' cst_int ',' repl_op ')'
        |    'SELECT' '(' cst_expr ',' repl_op ',' repl_op ')'
        |    repl_iterop
        |    repl_modifier
        |    repl_apply
        |    repl_macro
    Copy to clipboard

`OUTPUT_OF( "operand_tag", int_expr )` can be used, in some contexts,
to obtain a specific output of a multi-op output.

`WrapOp("opname", some_op)`, which allows exactly one input to `"opname"`, works by evaluating `some_op`, and then constructing `Op("op_name", some_op)` with the same Dtype and output as `some_op`. The Op Id and split-history are inherited in the usual way; it is generally equivalent to `WITH_SAME_OUTPUT(X, Op("opname", X))` but without needing to evaluate `X` twice. However, if `some_op` is an `"opname"` Op, it does not construct a new Op, it evaluates to `some_op` (on the assumption that `Op("op_name", Op("op_name",x))` is equivalent to `Op("op_name",x)`. When this assumption is *not* true, use `WrapOpAlways` which has the same behaviour, but will always add the new Op.

`repl_op` listed here under `repl_iterop` may only be used within
the last parameter of an `AUTO_SPLIT` or `OP_ITER`, and must
reference the same split\_tag.

repl_iterop ::=
            'ITER_INPUT_OF' '(' cst_opref ',' split_tag ')'
        |   'AUTOSPLIT_SHAPEFN_APPLY' '(' <name> ',' split_tag [',' apply_parm ]* ')'
    Copy to clipboard

*Modifiers*:

A modifier sets the attributes of the output tensor of any `Op()` in
its last operand, according to its preceding operands. So, the last
operand must be a <cite>repl_op</cite> which constructs an Op (including, another nested modifier).

For `WITH_OUTPUT_TYPE`: the `cst_expr` are `dtype`, `zero_off`
and `stepsize`and must be of type
`dtype, int, floa`t.`ResizeDim` changes one dimension to a a
given value; the first two params are the dimension number and the value
to change to.

repl_modifier ::=
            'WITH_SIZE' '('  repl_op  ','  repl_op ')'
        |   'WITH_TYPE' '('  cst_opref  ','  repl_op ')'
        |   'WITH_SAME_OUTPUT'  '('  cst_opref  ','  repl_op ')'
        |   'WITH_OUTPUT_TYPE' '(' cst_expr ',' cst_expr ',' cst_expr ','  repl_op ')'
        |   'WITH_MULTI_OUT' '(' cst_expr ',' repl_op ')'
        |   'ResizeDim' '('  cst_int ','  cst_int  ',' repl_op ')'
        |   'WITH_SAME_ID' '(' cst_opref  ','  repl_op ')'
        |   'WITH_SPLIT_HISTORY' '(' cst_opref  ',' cst_expr ',' repl_op ')'
        |   'WITH_SPLIT_HISTORY' '(' cst_opref  ',' repl_op ')'
    Copy to clipboard

(Currently, when `WITH_SPLIT_HISTORY` has 3 parameters, the second must be an integer constant)

The operations below call external functions.

`EXTERNAL_REPLACE` is typically used to implement the entire
replacement rule, as `EXTERNAL_REPLACE( funcname )`; where funcname is
a function of signature
`OpRef function(Replacement & rpx, const OpDef &oldop);`‘`oldop`’
is the `OpDef` being replaced. If the returned `OpRef` refers to
`oldop`, the graph is not modified.

`SHAPEFN_APPLY` is given a function of signature
`OpRef function(Replacement &rpx, ...args...);`where the ‘…args…’
are derived from the ‘`apply_parm`’. Operand names become `OpRef`,
and `cst_expr` become scalar expressions of the corresponding type.

`AUTOSPLIT_SHAPEFN_APPLY` is similar, but is given a function of
signature
`conv_split_start_valid(Replacement &rpx, Split_Context const &splitinfo, ...args...);`The
‘`splitinfo`’ is the context indicated by the ‘`split_tag`’, second
parameter of `AUTOSPLIT_SHAPEFN_APPLY`

The `SHAPEFN_APPLY` and `AUTOSPLIT_SHAPEFN_APPLY` functions
nominally return an `OpRef` representing a newly constructed
‘`OpDef_Shape`’; they may also return a `QuickShape` (which is a
data structure representing a shape in a more compact way), and the
framework will convert that to an `OpRef`.

External functions may also construct a Const data array, and return its `OpRef`.

These external functions need a annotation comment in the below format.
It is added for the benefit of the external parser.

// :::EXTERNAL_SHAPEFN::: { <return_type> function_name(<input_types>); }
    Copy to clipboard

See also:

- ref: `ShapeFnApply`
- ref: `AutoSplitShapeFnApply`

repl_apply ::=
            'EXTERNAL_REPLACE' '(' <name> ')'
        |   'SHAPEFN_APPLY' '(' <name> [',' apply_parm ]* ')'
    Copy to clipboard

In apply\_parm context, a `<string_constant>` is always a
`repl_op -> operand_tag-> <string_constant>;` other types of constant
are `cst_exp`.

apply_parm ::=
             repl_op
        |    cst_expr
    Copy to clipboard

The ‘`repl_macro`’ are equivalent to expanded expressions, as below.

However, in the case of AUTOSPLIT\_SLICE, TYPICAL\_SLICE, CHANGEDIM\_SLICE,
the implementation may construct the first operand only once, and AUTOSPLIT\_SLICE
may evaluate the ‘size’ input only once. So, these operands should not be an <cite>Op()</cite>.

See also :

- ref:`Replacement::AUTOSPLIT_SLICE`
- ref:`Replacement::TYPICAL_SLICE`
- ref:`Replacement::CHANGEDIM_SLICE`

repl_macro ::=
            'AUTOSPLIT_SLICE' '('  cst_opref ',' repl_op, ',' repl_op ')'
                    // AUTOSPLIT_SLICE( in, start,size) =>
                    //     WITH_SIZE( size, WITH_TYPE( in, Op("Slice_shape", in, start, size)))
        |   'TYPICAL_SLICE' '(' cst_opref ',' split_tag ')'
                    // TYPICAL_SLICE( in, tag )=>
                    //     AUTOSPLIT_SLICE( in,
                    //           AUTOSPLIT_SHAPEFN_APPLY( simple_split_start, tag, in ),
                    //           AUTOSPLIT_SHAPEFN_APPLY( simple_split_size, tag, in ))
        |   'CHANGEDIM_SLICE' '(' cst_opref ',' split_tag ',' new_dim ')'
                    // CHANGEDIM_SLICE( in, tag )=>
                    //     AUTOSPLIT_SLICE( in,
                    //           AUTOSPLIT_SHAPEFN_APPLY( simpledim_split_start, tag, in, new_dim ),
                    //           AUTOSPLIT_SHAPEFN_APPLY( simpledim_split_size, tag, in, new_dim ))
        |   'OpMultiOut' '(' cst_expr ',' cst_expr ',' opstring [',' repl_op]* ')'
                    // OpMultiOut( n_out, outno, "opstr", .. inputs .. ) =>
                    //           Op( "$Out", WITH_MULTI_OUT( n_out, Op( "opstr", .. inputs ..)),
                    //               gen_shape(0,0,n_out, outno));
    Copy to clipboard

**Restrictions** :

A Replacement rule should meet the restrictions below, most of which are not reflected in the grammar. Rules which
do not conform may still work, but may not continue to work in future implementations.

In the below, “existing opref” means a `cst_opref` as defined in the grammar;
Such an expression always refers to a node which is in the graph before the replacement rule is applied.

> 
> 
> - The first operand to `WITH_SIZE`, `WITH_TYPE`, `WITH_SAME_OUTPUT`, `WITH_SAME_ID`, `WITH_SPLIT_HISTORY` must *not* be an Op construction (defined below).
> It can be an existing opref; or in the case of `WITH_SIZE`, a shape construction (defined below).
> - The operand to `gen_ShapeOf` should be an existing opref. It can be a shape construction, but then the `gen_ShapeOf` is redundant.
> - The first operand to `AUTOSPLIT_SLICE`, `TYPICAL_SLICE`, `CHANGEDIM_SLICE` must be an existing opref.
> - The second and third operands to `AUTOSPLIT_SLICE` must be existing opref, or shape construction (or a `SELECT` which yields one of these).
> - The last operand to a modifier must be an Op construction.
> - The first operand of OP\_ITER must either be `Op(...)`,  or an existing opref. Note that when it is `Op(...)`, the constructed Op is always temporary, even
> when the iteration does not add additional inputs.
> - The entire replacement rule may not simply be `"*"`; and it may not be a `SELECT` or nested `SELECT` which is capable of evaluating to `"*"`.
> 
> 
> 
> 
> > 
> > 
> > It is permitted to be an existing opref other than `"*"`; or to be a `SELECT` which may evaluate to an existing opref; in these
> > cases the replacement rule just ‘bypasses’ some existing part of the graph; e.g the output of `x -> IntToFloat -> FloatToInt` might be
> > bypassed to just `"x"`.
> 
> 
> - `AUTOSPLIT` and `EXTERNAL_REPLACE` may only appear as the “root” of a replacement rule, or within a `SELECT` or nested `SELECT` at the root of a rule.
> 
> 
> 
> A ‘‘shape construction’’ is one of :
> 
> - <cite>gen_Shape</cite>, <cite>gen_ShapeOf</cite>
> - SHAPEFN\_APPLY or AUTOSPLIT\_SHAPEFN\_APPLY, for a function which returns a shape
> - `SELECT` where the second and third operands are both shape constructions.
> 
> 
> 
> An ‘‘Op Construction’’ is one of :
> 
> - `Op( ... )` or `OpMultiOut`
> - `WrapOp("name", x)` or `WrapOpAlways("name", x)`. It doesn’t make sense to wrap these in a modifier other than `WITH_SAME_ID` or `WITH_SPLIT_HISTORY`, unless `x` is an Op construction to which the modifier can apply.
> - `OP_ITER`,  `AUTOSPLIT_SLICE`,  `TYPICAL_SLICE`, `CHANGEDIM_SLICE`
> - Any modifier (which must itself contain an Op construction, as the last operand).
> - `SELECT` where at least one of the second and third operands is an Op construction.

Strictly speaking, `AUTOSPLIT` and `EXTERNAL_REPLACE` are Op constructions, but may not be used inside modifiers.

**Examples** :
We provided some common optimization utility functions usage examples, please check [here](https://docs.qualcomm.com/doc/80-63442-10/topic/common_optimization_utility_funcs_usage_examples.html#opt-utility-funcs-usage).

Last Published: Aug 06, 2026

[Previous Topic
QNN HTP Op Package API Revision History](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/opPackage_API_version_guide.md) [Next Topic
QNN HTP Op Package - Relu Op Example](https://docs.qualcomm.com/bundle/publicresource/80-63442-10/topics/relu_example.md)